Refute a step

/refute <entry> <step> dispatches one reviewer, who changes nothing. The reviewer does what a builder's report cannot do for itself: rerun the commands and reproduce the claims. It leaves behind agents/reviews/<step>-refuter.md: a verdict per item of the brief and per case, and a list of findings each with its place (a file and a line in code, a page and its section in a page) and its failure scenario, or "none" under a heading.

Quick start

/refute <entry> <step>   a fresh reviewer reads the step's diff, reruns the checks the brief's "Verify before you report" names and every quoted command, never the plan's whole verify list, and writes verdicts and findings

Use instead

What it reads

1. .agents/plan.yaml, its required keys and defaults as /plan states them.
   • A required key missing is a refusal ("Stops").
2. The ledger folder: <entry> resolves to the folder under <ledger_root>/ whose plan.md opens with # Plan: <entry> (the number, or the number and title).
   • /plan names a new folder by the entry's slug, and an older plan keeps whatever folder it has.
   • No such folder is a refusal ("Stops").
3. orchestrator-state.md: the dispatch block names the worktree, the base and the report path.
4. The brief agents/briefs/<step>.md, the rules file and the standards it points at, and the plan's text for the step.
   • The cases ruling agents/briefs/<step>-cases.md when one exists, read with the brief.
     • Where it rules a case, the diff is judged against the ruling.
5. The diff since the base, from inside the worktree, the new files whole, and a sample of a mechanical sweep with the sample named.
   • git diff <base>, git status --short and git show <base>:<path>, read-only, are the only git the reviewer runs.
     • git show <base>:<path> reads a file of the unchanged tree.
   • Then the ADRs the brief names under "What is on the tree", and every other NNNN-*.md record in the folder the configuration block's adr names (docs/adr when the block has none) whose part in force governs a file, a name, a rule or a behaviour the diff changes, in force and decision as the spec skill's "What it reads" 5 says.
6. The builder's report, last.
   • No report on disk is a refusal ("Stops").

Steps

1. Dispatch one reviewer as the effort agent ordo-<reviewer_effort> (the configuration block's reviewer_effort, high when the block has no such key), on the model the configuration block's reviewer: names, once per step before its first repair round.
   • A small text step gets this one run, whatever the configuration block's review: says.
   • A small text step has no run over a repair round.
   • Before the dispatch, check that the runner lists that agent among its agent types.
   • Before the dispatch, check that CLAUDE_CODE_EFFORT_LEVEL is unset: printenv CLAUDE_CODE_EFFORT_LEVEL exits 1.
   • Either check failing is the refusal "The configured effort cannot apply" ("Stops"), and no reviewer is dispatched.
   • Right after the dispatch, the orchestrator or the session reads the reviewer's agent id and the model the runner served it, from the runner's record of the agent as plan-orchestration's "Launching a builder" says.
   • A served model that is not the configured one is the stop "A model other than the configured one" ("Stops"): the reviewer is stopped through the runner's stop tool, and nothing it wrote is used.
   • The configured model is the model the configuration block's reviewer: names for the first run, and for a run over a repair round the model "Steps / Over a repair round" 1 gives.
   • The stopped reviewer is recorded under the reviewer_report of the step's dispatch entry, after the records before it.
     • A first-run reviewer: (<agent id>, <served model>, stopped).
     • A reviewer over round <n>: over round <n>: <agent id>, <served model>, stopped.
     • The record is written and carried as Steps 7 says.
   • Done when the reviewer is dispatched and its agent id and served model are read, or the refusal or the stop is raised.
2. The reviewer reads the inputs in the order "What it reads" gives them.
3. The reviewer runs the checks the brief's "Verify before you report" names, from the directory each names, piped through the filter the rules file names.
   • The lines they print are what the refuter report quotes.
   • The plan's whole verify list runs once, at landing on main.
4. The reviewer runs every command the report quotes as evidence, in the same form, and compares the output with what the report claims.
   • A quoted command that runs a git command other than the three of Rules is not run, and is named under "Declined to judge" as Rules says.
   • Where a claim needs a second build to reproduce (an A/B, a size figure), the reviewer says so.
     • It reproduces what it can from the one build.
5. The reviewer looks for the findings "The four headings" lists and gives the verdicts "The verdicts" lists, until every item of the brief's "What to build" and every case of its "Cases" has a verdict.
6. The reviewer writes the report from templates/report.md.
   • The verification lines first, verbatim.
   • Then the verdicts, as "The verdicts" says: one per item of the brief's "What to build", then one per case of its "Cases".
   • Then the four headings, each with findings, or "none", and each finding with its place (a file and a line in code, a page and its section in a page), the quoted hunk, what is wrong, its failure scenario as "The four headings" says, and the verdict it names when it has one, as "The verdicts" says.
   • Then "Declined to judge": each point the reviewer did not check, or declined because it is the user's call or outside what a read and a rerun can settle, with the reason.
   • The report's usage line stays as the template gives it, since the session fills it (Steps 7).
   • Done when the report holds the verification lines, a verdict for each item and case, the four headings and "Declined to judge".
7. The orchestrator or the session saves the report at agents/reviews/<step>-refuter.md.
   • It records the report's path under the reviewer_report field of the step's dispatch entry, after the records before it, followed by, in parentheses, the reviewer's agent id, its served model (Steps 1), and its tokens, tool uses and time from its completion notice: <path> (<agent id>, <served model>, <tokens> tokens, <tool uses> tool uses, <time>).
   • It fills the report's usage line with the same agent id, served model, tokens, tool uses and time, from the completion notice.
   • It checks the saved report with the character-set check.
     • A character outside the set is written in place by its name in angle brackets, such as <tab> or <em dash>.
   • Both are written to disk in the main checkout.
   • The next resume-point commit of its own plan carries them, and they get no commit of their own, as plan-orchestration's "Resuming, and handing the plan over" says.
   • Done when the report is saved with its usage line filled and its characters in the set, and its record is under reviewer_report.
8. Each finding is then closed or raised to the user, as "Finding dispositions" says.

Over a repair round

1. When the configuration block holds refute_after_repair: yes, /refute runs again after each of the step's repair rounds, a small text step having none (Steps 1), at most repair_rounds, or one more under plan-orchestration's exception, on a fresh reviewer each time, as Rules 1 says, dispatched as Steps 1 says except for its model.
   • The model is the one the configuration block's repair_reviewer: names, or the reviewer: value when the block has no repair_reviewer: key, at the effort reviewer_effort names.
   • This holds for every run over a repair round, the run over the extra round of plan-orchestration's exception and a reviewer started over a round in place of one stopped for another model included.
   • A run that finds nothing ends the rounds.
   • Done when the reviewer of the run is dispatched on the model this item gives.
2. The reviewer reads the same files, plus the first refuter report, the dispatch block's round entries and the snapshot folder made when the round was sent, <worktree>-round-<n>, beside the worktree, with its files.txt.
3. The round's delta is, read against the whole diff since the base:
   • for each path of files.txt, the difference between the snapshot's copy and the worktree's file (diff -u), a deleted path against nothing;
   • for each path changed now that files.txt does not list, its change since the base, the part of git diff <base> for that path, or the whole file when it is untracked.
4. It looks for the same four things over that delta, and for every closure the builder claims:
   • a finding closed by removing a check rather than fixing what the check guarded;
   • a fix that reaches beyond the finding;
   • a claim of closure the reviewer's own rerun does not reproduce.
5. It reruns the checks the brief's "Verify before you report" names again.
6. The orchestrator or the session appends the run's verdicts, findings and points declined to judge to the same file under "Repair round <n>, refuted", in the shape templates/report.md gives it.
   • It fills the run's usage line from the completion notice, as Steps 7 says.
   • It checks the appended text with the character-set check, as Steps 7 says.
   • It records the run in the same field of the step's dispatch entry, after the records before it, as over round <n>: <agent id>, <served model>, <tokens> tokens, <tool uses> tool uses, <time>, written to disk and carried by the next resume-point commit of its own plan as Steps 7 says.
   • Done when the run's section is appended and its record is in reviewer_report.
7. The findings of the run over the last round are never sent to the builder.
   • Each is fixed at landing when it is small and inside the brief, or raised to the user as "Finding dispositions" says.
8. With refute_after_repair: no these runs do not happen.
   • The orchestrator's read of the delta stands in for them.

The four headings

• Spec. A finding is:
  • an item the report marks DONE whose diff does not do what the brief's fix text says;
  • a change no item asks for (name the item you would expect, or say "no item");
  • a substitute mechanism where the brief named a shape;
  • a decision the brief reserved for the user, taken;
  • a dependency the diff adds that the brief does not name;
  • a premise in the brief's "What is on the tree" section that the reviewer's own grep does not reproduce;
  • a change that contradicts the part in force of an ADR, with the ADR's number and the sentence of its decision quoted, and whether the brief asked for it;
  • an ADR the diff is under that the brief's "What is on the tree" does not name;
  • a case of a code step in the brief's "Cases" that the rules file's test rule calls a test for and no test of the step checks;
  • a case whose first run on the unchanged tree the report does not give;
  • a script written for the step, by the builder or by the session that runs it inline, that the brief does not build under "What to build" and that no line of plan.md's Rulings approves;
  • a step whose brief's size line says it is a small text step and whose diff changes a script, a test or a configuration file, or 20 lines or more, counted as the glossary's entry small text step says.
    • Such a step is then handled as a full step.
• Proof. A finding is:
  • a "seen failing first" claim with no quoted failing check;
  • a test that asserts a known defect as the expected result;
  • a threshold, tolerance or predicate widened;
  • a check made to pass by copying or exempting;
  • a static, thread_local or file-scope mutable added;
  • a null guard, an early return or a fallback standing where a fix was asked for;
  • a count, a path or a measurement in the report that the reviewer's own run does not reproduce, when a decision rests on it, and the finding names that decision;
  • a new or changed test of a behaviour the change adds or changes with no failure quoted for it from its run on the unchanged tree before the code was changed;
    • An edit to the test after that run is judged by reading it, and the reviewer asks for no further run on the unchanged tree.
  • a new or changed test of a behaviour the change preserves with no passing run quoted for it after the change, or before the change on the unchanged tree where it could run there;
  • a test that would still pass with the behaviour it is written for taken out of the code, found by reading it (an assertion over source text, over a label alone, over a constant, or over an effect the test environment never runs).
• Standards. A finding is:
  • a documented standard the diff breaks, citing the standard's file and rule;
  • a comment that carries history (a step or item number, a date, what the code did before);
  • non-ASCII;
  • a secret left unredacted in a line the builder's report quotes, under the rules file's rule on secrets in quoted command output;
  • a public surface changed without its page;
  • a sentence in a document, a head comment or a rules file that the diff makes false, found by grepping each name the diff changed across the documents and the comments;
  • a file over the size limit;
  • a rule of the repository's checks that the diff satisfies only because the check does not read that path yet;
  • a test of behaviour whose failure costs nothing (neither lost work, nor a broken installation, nor a wrong configuration accepted), under the rules file's rule that a test exists only for behaviour whose failure costs something;
    • Such a test is not a Proof pass.
  • code larger than its job: a branch, an option or a case for an input that has not happened and whose wrong answer costs nothing, judged by reading.
• Behaviour. A finding is a host- or user-visible change the report does not state, or states without the before and after.
• Each finding, under any of the four headings, carries its failure scenario: the concrete input or state and the wrong result it gives, or, for a finding in text, the reader and what the text leads them to do wrong.

The verdicts

• Items. One verdict per item of the brief's "What to build", in the brief's numbering:
  • holds: the diff does what the item's text says, with the evidence named;
  • violated: the diff does not do what the item's text says, with the finding under its heading named;
  • not applicable: the item does not apply to this tree, with the reason.
• Cases. One verdict per case of the brief's "Cases":
  • met: the test, the run the report quotes or the reading gives the expected result;
  • partial: the test, the run or the reading gives part of the expected result, with the missing part named;
  • unmet: the test, the run or the reading does not give the expected result;
  • not verifiable: neither a test, a run nor a reading can settle the case here, with what would settle it.
• A verdict of violated, partial or unmet always has a finding under one of "The four headings", and the verdict and the finding name each other.
• Over a repair round, the reviewer gives the verdicts again for the whole diff since the base.

Finding dispositions

• A finding is closed by the builder in a repair round (at most repair_rounds, or one more under plan-orchestration's exception), or at landing, or raised to the user as an open item in the state file, as plan-orchestration's Stops section says.
  • It becomes a step only as plan-orchestration's "What earns a step of its own" says.
• A contradiction of an ADR that the brief asked for is a rule clash: it is raised to the user as an open item, never closed in a repair round or at landing, since only the user rules between the step and the ADR.
• A contradiction of an ADR that the builder made against the brief is closed like any other finding, by a change that follows the ADR.
• A finding of an unapproved script is never closed by naming the script in a report, nor fixed at landing.
  • It is raised to the user as an open item, and nothing the script produced is used until the user rules.
• A finding of a small text step's review is never sent to the builder.
  • It is fixed at landing when it is small and inside the brief, and otherwise raised to the user as an open item.
• The open items hold only what the user must rule on.
• After the last round, the run's findings (or, with refute_after_repair: no, the orchestrator's read of the delta) are appended to the report, each finding's disposition under the Closed heading.
• /land refuses while a finding is left neither closed nor raised as an open item.

Stops

The first row is a stop, a decision for the user: it leaves an open item, booked in the state file's open items. The rows below it are refusals, which name their cause and leave nothing.

Anti-patterns

Rules

• The reviewer is a fresh session or agent every time, for the first run and every run over a repair round: never the builder, and never the session that wrote the brief when another is available.
• The reviewer never writes into the ledger itself.
• The reviewer invokes no skill: it reads the inputs "What it reads" lists, runs the commands this skill names, and writes its report itself.
• The reviewer starts no agent: every read and every command of the review runs in the reviewer's own session.
• The reviewer runs no git command other than git diff <base>, git status --short and git show <base>:<path>; a fact only another git command would give, such as what git ignores or the tree after staging, is named under "Declined to judge" with the command that would settle it.
• The reviewer writes its report in the rules file's character set, a character outside it written by its name in angle brackets, such as <tab> or <em dash>.
• The reviewer writes <REDACTED> in place of the value of a secret in every line it quotes, a verbatim one included, as the rules file's rule on secrets in quoted command output says.
• An unreproduced claim is a finding, except a count, a path or a measurement no decision rests on, as the Proof heading says.
• A time box, the configuration block's review_minutes when above 0 or one the invocation names, is respected by reporting what was checked and naming what was not.