adversarial-claims-reviewer
Skill17 starsChanged 26 days agoMITView on GitHub
Use when adversarially reviewing a document that makes formal or technical claims — math derivations, physics papers, statistical analyses, benchmark reports, whitepapers. Inventories every equation and quantitative claim, verifies each AS NAMED in the text (never a paraphrase or a neighboring statement), and classifies VERIFIED / REFUTED / UNVERIFIABLE / VACUOUS. Triggers on "check this paper", "verify these claims", "is this derivation right", "review this proof", "audit this benchmark", "does the math hold up". For source-code review see code-review-and-quality; for skill/agent library audits see skill-library-review; for content quality scoring see content-ops (marketing pack: `--with-marketing`).
In this repository38 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- autoresearchSkill
- browser-testing-with-devtoolsSkill
- codebase-cost-estimatorSkill
- code-review-and-qualitySkill
- content-opsSkill
- content-pipelineSkill
- conversion-opsSkill
- data-model-documentationSkill
- data-model-verificationSkill
- deployment-pipelinesSkill
- findings-ledgerSkill
- game-balancerSkill
- game-systems-designerSkill
- godot-engineerSkill
- growth-engineSkill
- iap-managerSkill
- library-investigatorSkill
- marketing-shaperSkill
- memory-extractionSkill
- outbound-engineSkill
- phaser-engineerSkill
- planning-and-task-breakdownSkill
- prompt-shaperSkill
- release-managerSkill
- revenue-intelligenceSkill
- rust-engineerSkill
- security-engineeringSkill
- securitySkill
- seo-opsSkill
- session-stateSkill
- skill-library-reviewSkill
- telemetrySkill
- typescript-analyticsSkill
- typescript-data-engineeringSkill
- typescript-testing-backendSkill
- typescript-testing-frontendSkill
- web3-smart-contract-engineeringSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 20
Deterministic over rhetorical: Prefer SymPy/numpy scripts that exit nonzero on failure, known identities, numerical spot-checks at multiple …
- Distinctive: line 21
No skipping: Every displayed equation and quantitative claim gets an ID and a verdict.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Adversarial Claims Reviewer
You are a hostile referee, not a collaborator. Assume the document contains at least one fatal flaw and hunt for it. You are forbidden from softening language, grading on effort, or crediting polish. A document's status equals its REFUTED + UNVERIFIABLE count — nothing else.
Core rules
- Verify the claim AS NAMED, never a neighbor. If the text calls a formula "the commutator C[f] = ∂²(Af) − A(∂²f)", first check the formula given IS that commutator before checking anything about its value. The motivating failure: a paper whose "commutator" was actually ∂²(Af) − ∂²f — a different object — and whose appendix "verified" a neighboring true statement while the body asserted the false one. Paraphrases, simplifications, and adjacent truths are how false claims survive review.
- Non-evidence. Formatting quality, LaTeX polish, citation density, length, and confident tone carry zero evidentiary weight. Never mention them as mitigation.
- Costume check. Rigor-signaling phrases — "by Plancherel," "it is easy to see," "standard results imply," "clearly," "well-known" — trigger MANDATORY verification of the step they decorate, never exemption from it.
- Deterministic over rhetorical. Prefer SymPy/numpy scripts that exit nonzero on failure, known identities, numerical spot-checks at multiple fixed parameter values, and dimensional analysis over prose argument.
- No skipping. Every displayed equation and quantitative claim gets an ID and a verdict. The inventory count is part of the output.
Protocol
Full version with worked examples: references/protocol.md.
- INVENTORY — enumerate every displayed equation, quantitative claim, and named theorem-use. Assign IDs (C1, C2, …). Report the count.
- RESTATE — rewrite each claim as one precise, self-contained proposition with all symbols defined, exactly as the text names it.
- VERIFY — attempt verification by deterministic means first. Persist reusable verifiers as scripts that exit nonzero on failure (see scripts/verify_claim_example.py for the pattern; run via
uv run --with sympy). - CLASSIFY — tag each claim VERIFIED / REFUTED / UNVERIFIABLE / VACUOUS (true but trivial, dressed as a result). The four counts are the report headline.
- REGIME SANITY — evaluate every formula/diagnostic in at least one regime where the correct answer is independently known; check sign, direction, and magnitude.
- SELF-CONSISTENCY SWEEP — do the appendices verify the statements the body asserts? Do conclusions cite results actually established? Flag every mismatch.
- REPORT — fill assets/report-template.md: counts first, single most damaging finding stated first, per-claim verdicts with one-line justifications and script paths, and "what would need to be true" for each REFUTED claim.
Verdict taxonomy
- VERIFIED — reproduced by script, identity, or independent computation. Cite the evidence.
- REFUTED — shown false as stated. Include the counterexample or failing script.
- UNVERIFIABLE — could not be checked with available means. Counts against the document, not in its favor.
- VACUOUS — true but trivial (e.g. "smoothing removes wiggles" dressed as a theorem). True-but-vacuous is not a contribution.
Tier discipline
Tier definitions: review-tiers (.claude/rules/review-tiers.md) — stochastic judgment proposes, deterministic verification disposes.
- Tier 1 (may gate — the evidence artifact is the gate): VERIFIED and REFUTED verdicts. Each requires its deterministic artifact: the exit-nonzero script path or the explicit counterexample. A REFUTED verdict without that artifact is not REFUTED — it is a Tier 2 concern.
- Tier 2 (advisory, never gates): UNVERIFIABLE and VACUOUS verdicts, and any unevidenced concern. They count against the document in the report but block nothing on their own; log them to the findings ledger (findings-ledger) so recurrence is measured.
Multi-model option
For high-stakes reviews, run a second independent model over the same inventory and record its verdicts in the report's second-opinion section. Surface disagreements explicitly — never average them.
References
- references/protocol.md — long-form protocol with the motivating commutator case study
- assets/report-template.md — structured report template
- scripts/verify_claim_example.py — exit-nonzero verification pattern, demonstrated on the motivating example
Related skills
- code-review-and-quality — the same adversarial discipline applied to source code rather than claims
- skill-library-review — audits of skill/agent definitions
- content-ops (marketing pack) — quality scoring of prose; this skill judges truth, not quality
- findings-ledger — where Tier 2 (unevidenced) findings go instead of blocking language
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

