adversarial-diff-verify
Skill0 starsChanged 3 months agoMITView on GitHub
Check a builder agent's structured claim against the file and its actual git diff, confirming or refuting each assertion independently. Use when a subagent reports "done" with a JSON or bulleted claim about what it changed, when Shane says "verify that conversion", "check what the builder actually did", "adversarially verify this diff", or after any fan-out where agents edited files in parallel. Do NOT use for reviewing code quality (use a code-review skill), for verifying a factual claim about the world (use /research-sweep's verify stage), or when there is no diff to read.
In this repository11 other files it gives its agents.
- agent-harnessSkill
- agents-md-generatorSkill
- plugin-manifest-referenceSkill
- publish-postSkill
- research-sweepSkill
- workflow-designSkill
- workflow-feature-specSkill
- workflow-monthly-skill-auditSkill
- workflow-stage-draftSkill
- workflow-vault-weeklySkill
- work-trackingSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Read what the builder actually did, not what it said it did — a claim and a diff disagreeing is the single most common failure in parallel …
- Distinctive: line 17
If either is missing, ask.
- Distinctive: line 21
Read the file IN FULL: Not a grep, not a range. Preservation claims are unfalsifiable without the whole text.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Skill: /adversarial-diff-verify [file] [builder claim]
Read what the builder actually did, not what it said it did — a claim and a diff disagreeing is the single most common failure in parallel agent work, and it is invisible unless someone reads both.
Don't: accept a claim because it is detailed and confident; the fabricated ones are the most detailed. Don't read only the diff — a diff shows what moved, not what the file now says, and claims about preservation are only checkable against the whole file. Don't return a verdict per file; return one per assertion, because builders are usually right about most of them and wrong about one.
Inputs
file— the path the builder edited.builder claim— its structured report. Typically JSON with fields likechanges[],contentPreserved,buildPassed, plus task-specific assertions.
If either is missing, ask. Do not infer the claim from the diff — that defeats the check.
Steps
- Read the file IN FULL. Not a grep, not a range. Preservation claims are unfalsifiable without the whole text.
- Read the whole diff:
cd <repo> && git diff -- <file>. If the builder already committed, usegit show <sha> -- <file>. - Decompose the claim into assertions. Each one gets its own verdict. A typical builder report yields 5–15.
- Test each assertion against the evidence, in this order of suspicion:
- Preservation claims ("kept verbatim: …") — check each named item is present and unchanged. This is where claims fail most often.
- Negative claims ("no new fact, no invented number", "zero figures shipped") — these need a search to confirm, not a reading. Run the grep. A negative asserted without a grep is unverified, not true.
- Build/test claims (
buildPassed: true) — re-run it. Do not take the flag. - Change claims ("wrapped in the shell, added X") — confirm the diff actually contains them.
- Reasoning claims ("every figure type fails its entry requirement") — check the reasoning against the stated rule, not against plausibility.
- Return a verdict per assertion:
confirmed,refuted, orunverifiable— with the evidence line or diff hunk that decided it. When uncertain,refuted, matching the fact-check default.
Output
<file>
confirmed: <n>
refuted: <n> ← list each with the contradicting evidence
unverifiable: <n> ← list each with what would settle itLead with the refuted ones. A clean run reports zero refuted and still lists what was unverifiable — silence on unverifiable assertions reads as confirmation.
Fan-out
One verifier per file, dispatched in parallel, each blind to the others' results. Do not let one agent verify several files — it starts pattern-matching the second file against the first and stops reading.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

