refutation
Skill0 starsChanged 54 days agoApache-2.0View on GitHub
Adversarial refutation-by-majority protocol (M13). Dispatch N independent read-only skeptics with an explicit REFUTE mandate per high-stakes finding; a finding survives only with a strict UPHELD majority. Persist skeptic verdicts as E1 agent-return envelopes and write the refutation-required marker so scripts/check-refutation-verdicts.mjs can adjudicate.
In this repository23 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .claude/CLAUDE.mdCLAUDE.md
- llms.txtllms.txt
- architect-reviewSkill
- brainstormingSkill
- clean-codeSkill
- close-gold-gapSkill
- codebase-auditSkill
- configureSkill
- context-rot-managementSkill
- epic-decomposeSkill
- gold-auditSkill
- impactSkill
- levelupSkill
- product-acceptanceSkill
- senior-surveySkill
- ssot-navigationSkill
- tabletopSkill
- tddSkill
- understand-codeSkill
- verificationSkill
- visual-verificationSkill
- wave-drainSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 0 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 29
Threshold: Default severity threshold is high (critical + high).
- Very specific: line 30
N is declared, not improvised: Read N from .claude/agent-dispatch-matrix.json refutation_skeptics for the active tier:
- Very specific: line 75
When a mechanical matcher — rather than this gate's structural check — has to decide whether LLM-authored text matches a ground-truth item, …
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Refutation-by-majority (M13)
High-stakes findings (audit findings at or above a severity threshold, structural claims destined to be acted on) are not accepted from a single agent. Independent skeptical agents are dispatched with the explicit mandate to REFUTE the finding against the actual code; a finding survives only if it withstands a strict majority.
Why
Single-reviewer verdicts inherit the reviewer's blind spots and a "reviewing to approve" bias. Refutation-framing plus independence is the cheapest known de-biaser; majority survival bounds both false positives (R4 — the "hexagonal architecture is fiction" incident was the fiction) and rubber stamps (R2).
Dispatch protocol
- Threshold. Default severity threshold is
high(critical+high). Lower only for audit modes that explicitly demand it. N is declared, not improvised. Read N from
.claude/agent-dispatch-matrix.jsonrefutation_skepticsfor the active tier:tier N XS 1 S 1 Standard 3 Solo XS/S = 1 skeptic (the red-team IS the skeptic). Standard = 3. Gated-review raises the floor to the full vertical set (matrix
tier_verticals.Standard).- Independence. Spawn N read-only skeptics (registry ladder per
.claude/AGENT_REGISTRY.md; rule-50 read-only clause makes them legal in parallel without worktrees). Each skeptic receives:- only the finding text + its citations — no sibling verdicts, no orchestrator opinion;
- the mandate: "your job is to REFUTE this claim against the actual code; return UPHELD only if you fail to refute it."
- Persist. Each skeptic's return is an E1 agent-return envelope (
role:"skeptic"+refutations[]), piped throughnode scripts/record-agent-return.mjs --task '#NNN'. Marker. The orchestrator writes
.arbiter/evidence/agent-returns/<task>/refutation-required.jsonat dispatch time — before the skeptic verdicts come back, as a numbered step of the review phase. The gate below adjudicates only when the marker exists, so leaving it to post-hoc discretion turns the gate into one that never fires. When the review produced no findings at or above the threshold, still write the marker with an emptyfindingsarray: that records the threshold was evaluated, instead of leaving "no marker" ambiguous between "nothing to refute" and "step skipped".{ "task": "#NNN", "threshold": "high", "skeptics": 3, "findings": ["f1", "f2"] }findings= the ids requiring refutation (the acted-on set). The skill refuses to conclude whilescripts/check-refutation-verdicts.mjsis non-zero.
Majority rule
UPHELD > N/2 ⇒ the finding survives and may be acted on. Otherwise it is demoted to info and logged — never silently dropped (M14 discipline: a refuted finding is still a recorded finding, just not an actionable one).
The loop: hop until nothing above low survives (INV-145, CANON-24)
One pass is not the protocol. After the skeptics report and their findings are fixed, dispatch a fresh round against the fixed tree — the new hop attacks the fixes, not the original claims — and repeat until no finding above low remains unaddressed. Empirically (#2480) the second hop destroyed the first hop's fixes twice: a gate that passed round 1 was still scoring a fully commented-out document 12/12.
Each hop is independent by construction: new agents, disjoint scopes, prompts naming the ARTIFACT and never the previous round's reasoning. Reusing a skeptic, or handing one the last round's rationale, converts refutation into confirmation.
Prefer the cross-model seat. When crossModelReview is configured (arbiter.json, provider e.g. codex), the hop runs there: a different model is a materially stronger skeptic than the same model in a different costume, because the blind spots are not correlated. The same-model fan-out below is a fallback, and it is weaker in exactly that way.
A hop that cannot run is recorded, not skipped. Model unavailable, rate limit, cross-model seat offline: self-probe the round's questions and write "degraded": true on the marker. The gate accepts it and says DEGRADED on every run. Never file a self-probe as an independent round.
Gate
scripts/check-refutation-verdicts.mjs (advisory runWarnCheck at L2+, promoted to runCheck at gated-review) adjudicates two axes over the same envelopes:
- Majority — every finding in
findingsmust have ≥ N skeptic verdicts and a strict UPHELD majority. Stops a phantom being acted on. - Severity floor (INV-145) — every finding the skeptics majority-UPHELD at
critical/high/medmust appear infindings. Stops the loop ending while something real is open. Severity is the highest any skeptic assigned. Below quorum or majority-REFUTED never blocks: one false alarm must not hold a wave hostage.
Marker present ⇒ both axes run, including when findings is empty — a round that addressed nothing while a high finding stood is precisely what the floor catches. No marker ⇒ the gate passes vacuously — which is exactly why step 5 makes writing the marker (empty findings included) the orchestrator's unconditional duty at dispatch: the gate can only adjudicate what was declared, and /ship and /drain both name this step in their review phases.
When a mechanical matcher — rather than this gate's structural check — has to decide whether LLM-authored text matches a ground-truth item, the sampled-audit-then-judge protocol in docs/internal/METHOD/ADJUDICATION.md applies.
Tier right-sizing
Fan-out never exceeds min(--max-parallel, nproc-2) (wave-drain cap). Solo never pays N=3 for a typo fix — N=1 is the red-team. Standard pays 3; gated-review pays the full vertical set. N is a declaration in the dispatch SSOT, not a runtime guess.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

