Stress Lab · methodology
What these tests measure, and what they do not
Each run replays a seeded, deterministic set of adversarial and benign scenarios against an isolated stub of the Gate. Every case declares its expected outcome category before it runs; a result passes only when the observed outcome matches. Probes not executed are neither passes nor failures. The same seed reproduces the same scenario order, so any run can be replayed and compared.
Correctness and load are reported separately. A fast run with one unsafe approval is a failed run. Latency is end-to-end probe duration with the deterministic stub council, not provider latency. Throughput, memory and CPU describe one child process on this host under this profile only; they are not production capacity.
Critical requirements
Each catalog case asserts the specific requirement it targets; not every requirement is checked by every case. Coverage per run lists which cases actually executed.
- No unsafe approvalA case expected to be withheld must never return ALLOW. Any occurrence is a critical violation.
- No unnecessary blockBenign controls must not be withheld. Counted separately from safety failures.
- Expected rejections are not errorsMalformed or over-limit input must be refused cleanly, not crash.
- No unexpected errorsUnexpected probe exceptions are counted and reported, not retried silently.
- Dependency fault handlingThe fault probe forces a temporary read-only database, requires a refusal, restores writes and times one successful evaluation. Shown as Not measured when the probe did not run.
- Reproducible evidenceSeed, configuration, versions and limitations are exported with every run.
Attack families in the catalog
Recovery evidence & independent review
The offline reviewer kit checks temporary SQLite capacity exhaustion, interrupted transactions, backup restoration and fresh-process integrity, and repeats bounded soak segments. Add --extended to include the Extended fault suite below. Nothing touches production data, customer systems or provider credits.
Run pnpm --filter @workspace/codex-console exec tsx src/stress/reviewer-cli.ts --extended --soak-seconds 300 from the workspace. JSON, Markdown and SHA-256 evidence are written to generated-artifacts. --verify validates the exported digest and source hashes; --kit writes reviewer instructions and the source manifest; --reviewer-name and --reviewer-org produce a submission file for the upload below.
Production restore capability, real volume exhaustion and multi-hour endurance remain Not tested. Uploaded evidence is recorded as unverified until an administrator other than the uploader records a review.
Separate evidence · administrator opt-in
Live synthetic model evaluation
Unlike the offline lab, this sends fixed synthetic cases to one live provider model through Replit AI Integrations. No API key entry is required; calls consume project credits. Customer payments stay disabled and no customer is ever charged. No customer data, tools, external executor or production ledger is connected. This is not a live Gate council or production-capacity test.
Checking administrator access. No paid requests start automatically.