Skip to main content
Conformance is the answer to “is this region actually real?” It is a single suite that runs against a live region over HTTP, asserting real behavior — and, crucially, refusing to fake anything it cannot really do.

​
Capability gating (KEYS.md)

Every leg that needs a real third-party key declares it with hasCapability(capability, [VARS]). When a var is missing the suite prints a loud, named error and skips that leg as BLOCKED — it is never mocked:
KEYS.md is the single source of truth for what each key unlocks and what happens when it is absent. The behavior-when-missing is always a hard, named skip, and the suite prints exactly which capability is dark and why.
A missing sandbox key does not license faking. The never-stub invariant applies to the enforcement path and the claim flow’s structure. Current BLOCKED capabilities (per KEYS.md): stripe-test-key, footprint-sandbox, comms-test-keys — the staging secret holds live keys, which are refused for sandbox/demo use. Synthetic sandbox adapters are the documented sandbox behavior of a primitive, not mocks of real calls.

​
What it asserts

The groups (ci/conformance/*.test.ts) cover health, auth, tenant users, the vault (put / masked / reveal), approvals (freeze → approve → replay via the governor), the queue (std / FIFO / DLQ), LLM, events (SSE), environments (incl. stamp hygiene), devices (claim e2e + relay cursor resume), and inertness. Every assertion also validates the response body against a JSON schema with ajv, so payload drift is caught per-request (route-set drift is the spec-drift guard’s job, below). The inertness group is run with the governor actually stopped: a gated action must return 403 enforcement_unavailable (asserted for real), while pure reads stay live. This is what proves enforcement is in the closed engine and the region fails closed without it.

​
The legs

The harness selects a leg with CONFORMANCE_LEG:

local-dev

The default, fast tier. ci/conformance-local.sh boots the cp-dev stack (Postgres + governor + api on the file / pg / fs drivers), runs the groups, then stops the governor and runs the inertness group before restoring the stack.

tower-vm

The same suite pointed at a booted golden image (CONFORMANCE_LEG=tower-vm) — the vaultd / pg / fs production substrates. This is the highest-fidelity run: real image, real topology, real TPM-sealed /data.
A cloud-staging leg (staging creds) exists for cloud-substrate parity. Because the drivers share one interface, the same assertions pass on every leg — that identity is the point.

​
The spec-drift guard

ci/spec-drift.test.ts keeps the hand-maintained openapi.json honest, in both directions against the real Express app:
  1. every route the app serves appears in the spec (no undocumented routes);
  2. every path in the spec is served by the app (no phantom paths).
A small, justified allowlist covers non-API surface (health/readiness, inbound provider webhooks, shared-secret cron sweeps, MCP transport, browser-facing redirects, the device claim/relay ingress). Everything else must match. The spec is load-bearing — it feeds typegen and the conformance schemas — so drift is a CI failure, not a lint.

​
The two-tier CI

The suites split by cost so most changes get fast feedback:
  • Fast tier — unit suites, typecheck, spec-drift, the docs assertions (ci/docs-assertions.sh), and the local-dev conformance leg (conformance-local.sh). No KVM, no image build.
  • Checkpoint tier (Linux + KVM) — the image build, the compose topology test (the four load-bearing assertions), the tower-vm boot gate, and the tower-vm conformance leg. These need a real Linux kernel (for runsc) and hardware virtualization (for a fast QEMU boot), so they are authoritative in CI rather than on a dev Mac.
The goal is that both tiers are green on a fresh clone. The fast tier gates every push; the checkpoint tier gates the artifacts that can only be verified on Linux/KVM.