Verification Oracle

Deterministic re-execution of a command or code gate: run the declared check again, observe whether it passes now, and return a verdict carrying the evidence that produced it. command and code are one mechanism with two labels -- both carry a runnable string, evidence_shape on a persisted gate or command on a contract's evidence_report.verification -- so the label changes what a reader expects (a suite versus a linter), not how the oracle runs it. For semantic and self_review gates, use verification-rubric.

Position in the flow

Upstream, the planner authored the gate: its claim in evidence_type, its check in evidence_shape. Upstream of that, the executor ran the check red before its change and green after, recording both as evidence tied to the gate. The oracle never relaxes the gate: it runs the declared check as written and reports whether it holds. Probes against the claim belong to the verifier (gaia-verifier), which runs them after this check. Downstream, its verdict decides whether the task derives as done and, through the evidence it leaves, whether an AC can close -- so a false pass here counts unfinished work as done one layer up.

Process

1. Confirm the type. Only command and code apply. Re-running a semantic rubric or a self_review statement as a shell command is a category error: there is nothing to execute.
2. Extract the check. Take evidence_shape (gate) or command (envelope). A blank check on a deterministic type is a failure of the gate itself, never an assumed pass.
3. Treat a stale verdict as pending. A gate whose stale_at is set keeps its old verdict for the record, but something it depended on changed since. Re-run it exactly as if no verdict existed.
4. Re-execute; do not re-read. Run the exact string directly as one command, never through a shell (command-execution): the Bash tool reports the exit code itself, so do not wrap the command in bash -c or append echo $? to capture it. Keep stdout, stderr and that reported exit code. Exit 0 is pass. Re-observing is the whole point: an oracle that accepts a prior claim is a self_review with extra steps.
5. Check that the gate could fail. For a gate on a change, look for its red run (gaia evidence list, negative evidence tied to this gate). A check that was never shown failing, or that passes whether or not the change is there, cannot distinguish done from not done. Then check that the selection was not narrowed since that red run. The shape lives in the gate row: its current evidence_shape carries no -k, -m or --deselect absent from the command recorded with the red-run evidence, and a stale_reason names any gate edit since. The test files live in git: git log -p over them from the red run's created_at shows no new skip or xfail. Added tests are growth; a narrowed selection is broken_test.
6. Return the full verdict and record it. Pass, or fail with the cause that says what has to move: product (the change does not do it), environment (the runner, a dependency, the network), broken_test (the check is wrong or cannot fail), requirement_changed (the claim no longer matches the brief). Keep the command, exit code, stdout and stderr for verbatim_outputs and as the gate's evidence. A timeout or a missing binary is a failure with its own error, never a silent pass.

Reference implementation

gaia.state.gate_oracle.run_oracle_check(gate, timeout=60.0) runs step 4: it accepts either the gate or the envelope shape and returns an OracleVerdict with ok, the resolved command, the exit code, stdout, stderr and errors. Given red_selection=gate_selection(command) captured at the red run, it also fails on step 5's narrowing (selection_narrowing); no CLI persists that snapshot yet, so the git log -p read stands in. Step 3 and the failure cause are the verifier's judgment on top of it.

Anti-patterns

• A second execution path for code -- both types resolve to the same runnable string; an exec() of a snippet diverges from what gaia.state.gate_validation settled and doubles what must be maintained.
• Passing a gate that never went red -- a check that cannot fail proves nothing about the change; that is broken_test, not pass.
• Trusting a stale pass -- the verdict predates the change that marked it; only a fresh run clears it.
• Widening the run -- the oracle re-runs only the declared gate and probes, never a wider run (code-standards Rule 9).
• Skipping re-execution because the producer said it passed -- that is what self_review is for; a command/code gate exists so someone re-runs it.