AGENTS.md — flow-corpus

The calibration population. Synthetic, deterministic, and firewalled from the harness by design.

This directory owns the flow specimens, task suites, oracles, mutation engine, version keying, holdout rotation, discrimination canary and validation runner. Its top-level facade exposes only the version pins; everything else is reached through a subpackage with its own contract.

Map

Diagram

flowchart LR
  accTitle: flow-corpus run lifecycle from suite to reliability
  accDescr: Pin verification gates the run; a suite instance goes through a specimen to a flow result, an oracle judges it, determinate verdicts become outcomes keyed by agent version and domain while indeterminate ones are counted and routed to audit, and the runner reports reliability.

  PIN["pinning.py<br/>verify_pins or raise"]
  SUITE["suites/ + data/"]
  SPEC["specimens/"]
  MUT["mutation/, canary/<br/>known signal"]
  FR["FlowResult<br/>contract type"]
  ORC["oracles/<br/>kappa-gated"]
  KEY["keying/<br/>(agent_version, domain)"]
  IND["indeterminate<br/>counted, never an outcome"]
  RUN["validation/runner.py"]
  REL["reliability + intervals"]

  PIN --> RUN
  SUITE --> SPEC
  MUT --> SPEC
  SPEC --> FR
  FR --> ORC
  ORC --> KEY
  ORC --> IND
  KEY --> RUN
  RUN --> REL

  classDef here fill:#e8f0fe,stroke:#1a73e8,stroke-width:2px
  class RUN here

Rules that bite here

• Never import eval_harness, in any module, for any reason. This package may reach flow_protocol and agent_core and nothing else. The absence of that edge in architecture.yaml is the airgap, and drift_check.py fails the PR the moment it appears.
• The version pins are a tripwire, not bookkeeping. PROTOCOL_VERSION_PIN and HARNESS_VERSION_PIN in version.py are compared against the live packages, and a mismatch raises PinMismatchError so the build dies before it produces keyed stats against an unexpected contract. When a bump breaks it, that is the gate working — adopt the new version deliberately, with a passing verify_pins(), rather than syncing the pin to make red go away.
• Outcomes are keyed by (agent_version, domain) and the task instance is deliberately not part of the key. Task variation is the population being measured over that key; folding it in collapses the very distribution the corpus exists to characterise.
• An indeterminate verdict is counted, never converted. The runner routes abstentions to the audit path and keeps them out of the outcome stream, so the gate is never fed a guess.
• A non-finite config value deletes a floor silently. Every comparison against NaN is false, so config.py rejects NaN and infinity up front — _require_finite exists because a NaN power threshold once made an under-powered sample gate-eligible. Keep new fields guarded.
• The facade stays narrow. Adding a convenience re-export to flow_corpus/__init__.py bypasses the subpackage contracts that each hold the real local constraint.

Verify

make -C flow-corpus check

Subagents

See also