claim-check
Skill4 starsChanged 6 months agoMITView on GitHub
Walk every substantive claim in a section, classify it (cited / author's-own / common-knowledge / UNSUPPORTED), verify the citation licenses the claim's verb and quantifier, and report supported/overstated/uncited counts
11 other projects use a version of this.
Adopted by
Most recently taken up by
- mbalazy/claude-skillstheir own version, first seen Nov 16, 2026
- Balance0014/claude-code-claim-checktheir own version, first seen Nov 16, 2026
- adithyanraj03/dsh-skills-packtheir own version, first seen Nov 15, 2026
- Job-Yang/jobbyang-ai-skillstheir own version, first seen Nov 8, 2026
- Job-Yang/jobyang-ai-skillstheir own version, first seen Nov 8, 2026
- microsoft/cat-agent-skillstheir own version, first seen Nov 8, 2026
Other versions11 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 11
- RyanAlberts/best-of-Agent-Harnesses★ 1.5ksame name
- microsoft/cat-agent-skills★ 108same name
- Job-Yang/jobbyang-ai-skills★ 100same name
- Job-Yang/jobyang-ai-skills★ 100same name
- Balance0014/claude-code-claim-check★ 2same name
- duu261/agent-skills★ 2same name
- NotRedFox/NotRedFoxs-Claude-skills★ 1same name
- 8TrafficAI/8traffic-marketing-skills★ 0same name
- adithyanraj03/dsh-skills-pack★ 0same name
- mbalazy/claude-skills★ 0same name
1 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository24 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- abstractSkill
- citation-auditSkill
- cover-letterSkill
- gap-finderSkill
- journal-fitSkill
- lit-briefingSkill
- literature-reviewSkill
- lit-ingestSkill
- lit-lintSkill
- manuscript-cycleSkill
- methods-reviewSkill
- noteSkill
- outlineSkill
- peer-reviewSkill
- plain-language-summarySkill
- reference-formatSkill
- response-to-reviewersSkill
- retroSkill
- review-resurfaceSkill
- scorecardSkill
- stats-checkSkill
- submission-pipelineSkill
The file
We split this skill into the rules it gives an agent
4 rules. 0 are common, 3 are distinctive, and 1 is very specific to this project.
0
Common
Most of the 8 edited versions we read keep these.
3
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 21
Before submission, on the Introduction, Results, and Discussion — the three sections where overclaim hides.
- Distinctive: line 22
After a revision that added claims, to confirm none slipped in uncited.
- Distinctive: line 30
Before reading a single claim, load what the section is allowed to assert:
- Very specific: line 37
If MANUSCRIPT_MAP.md → Key sources says a reference must not be overclaimed a certain way (e.g.
“Common” is measured: kept by at least 6 in 10 of the 8 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Claim Check
Core Rule
Every substantive claim is one of four things: cited, the author's own reasoning/data (stated as such), common knowledge in the field, or UNSUPPORTED. A claim that is none of these does not belong in the manuscript yet. Never fabricate a citation to "fix" an uncited claim — flag it with [CITE] and tell the author what is missing. A plausible-looking reference you cannot point to is a fabrication, not a fix.
This is the flagship skill. It is the manual, source-reading half of verification that a hook cannot do: citation-gate.sh proves a \cite key resolves to a .bib entry; only a reader can prove the entry licenses the sentence.
When to Use
Invoke with /claim-check when:
- A section draft is "done" and you want to know what a skeptical Reviewer 2 would attack.
- You inherited prose (yours or a co-author's) and need to know which sentences are load-bearing but unsourced.
- Before submission, on the Introduction, Results, and Discussion — the three sections where overclaim hides.
- After a revision that added claims, to confirm none slipped in uncited.
Scope it: /claim-check sections/discussion.tex for one section, or name a paragraph. Checking a whole manuscript at once produces a table too long to act on — go section by section.
Process
Phase 1: Establish Ground Truth
Before reading a single claim, load what the section is allowed to assert:
- Read
MANUSCRIPT_MAP.md— the Thesis, the Key sources table (what each.bibkey establishes and what it must NOT be overclaimed as), and "Claims that need extra care." - Read the target
.texfile in full. Do not skim — every declarative sentence is a candidate claim. - Read
references.bibfor the keys this section cites. Note each entry's actual scope: population, matrix, method, sample size, and the strength of its own language (did the source say "associated with" or "causes"?). - Open the sources in
sources/for the spine references. If a source PDF/note is not in the library, you cannot verify a claim against it — record that gap; do not assume the claim holds.
If MANUSCRIPT_MAP.md → Key sources says a reference must not be overclaimed a certain way (e.g. "single-turn QA baseline — NOT evidence for multi-turn agentic tasks"), that is a hard constraint for this pass.
Phase 2: Enumerate Claims
Walk the section sentence by sentence. A substantive claim asserts a fact about the world, the literature, or your results that a reader could dispute. Skip pure transitions, definitions you established, and signposting.
For each claim, record: the verb (asserts / shows / suggests / proves / causes / is associated with), the quantifier (all / most / often / in our sample / generally), and any number. Verb and quantifier are where overclaim lives — capture them precisely.
Phase 3: Classify Each Claim
Assign exactly one class:
| Class | Definition | Action |
| CITED | Carries a \cite{key} resolving to references.bib | Verify in Phase 4 |
| AUTHOR'S-OWN | Your data, reasoning, or contribution, stated as such | Check it matches your Results/data; no external cite needed |
| COMMON-KNOWLEDGE | Uncontested in this venue's audience; needs no cite | Confirm it is genuinely common for this audience (see MANUSCRIPT_MAP → Audience) |
| UNSUPPORTED | Disputable, not cited, not your data, not common knowledge | Flag [CITE] — needs a source or must be cut/hedged |
Discipline on the easy escape hatches:
- "Common knowledge" is audience-relative. For a specialist methods venue, a basic mechanism may be common; for a broad-readership journal it must be cited. When in doubt, it is not common knowledge.
- "Author's-own" must be visibly framed as such ("We observe…", "Our data indicate…"). An unframed assertion sitting next to cited sentences reads as cited — that is a sourcing ambiguity, flag it.
Phase 4: Verify Each CITED Claim Against Its Source
For every cited claim, check the citation actually licenses the verb and the quantifier:
- Verb strength — the source must support the claim's verb. If the source reports an association and the sentence says "causes," that is OVERSTATED. "suggests" ≠ "proves"; "is consistent with" ≠ "demonstrates."
- Quantifier / scope — "in general" needs evidence beyond one setting/harness/task type. A single-turn QA result does not license a multi-turn agentic claim. Scope creep is OVERSTATED.
- Quantity — any number must match the source exactly. A misremembered statistic is a fabrication even with a real cite. If you cannot confirm the number against the source, mark it
[VALUE — verify]; do not let it pass. - Attribution direction — the cite must support this claim, not a neighbouring one. Watch for a
\citethat backs the first half of a sentence while the second half (the actual claim) rides along unsupported.
Outcome per cited claim: SUPPORTED (verb + quantifier + number all licensed) or OVERSTATED (calibrate down).
Phase 5: Report and Prioritize Fixes
Produce the table and a prioritized fix list. Do not edit the manuscript in this skill unless the user asked — claim-check reports; fixing claims (especially adding/removing a cite that carries an argument) is a Protected Claim and needs author sign-off per CLAUDE.md.
Output Format
# Claim Check — sections/discussion.tex
## Summary
- Claims examined: 18
- Supported: 11
- Overstated: 4 (verb/quantifier exceeds the cited evidence)
- Uncited: 3 (UNSUPPORTED — flagged [CITE], NOT fabricated)
## Claim Table
| # | Claim (verb / quantifier) | Class | Cite | Verdict | Note |
|---|---|---|---|---|---|
| 1 | "the gate improves tool-call accuracy (in our sample)" | author's-own | — | SUPPORTED | matches Results Tab 2 |
| 2 | "post-hoc self-correction fails on long horizons (causes)" | cited | tooluse2023 | OVERSTATED | source shows association, not causation |
| 3 | "this is the first study of pre-execution gating" | author's-own | — | OVERSTATED | novelty claim with no comparison shown |
| 4 | "LLM agents can hallucinate tool calls" | common-knowledge | — | OK | uncontested for this audience |
| 5 | ">90% tool-call accuracy generally" | cited | tooluse2023 | OVERSTATED | tooluse2023 is single-turn QA; scope creep to multi-turn agentic tasks |
| 6 | "prior survey reports a 23% hallucination rate" | cited | halluc2022 | [VALUE — verify] | number not confirmed against source |
| 7 | "hallucinated tool-call rate scales with task horizon" | UNSUPPORTED | — | [CITE] | disputable, no source, not our data |
## Prioritized Fixes
1. **[Overclaim — Protected]** Claim 2: change "causes" → "is associated with" (tooluse2023 reports association). Needs author approval — verb change on a spine source.
2. **[Overclaim — scope]** Claim 5: restrict to "on single-turn QA (tooluse2023)" or cite a multi-turn agentic source. MANUSCRIPT_MAP flags tooluse2023 as NOT a multi-turn agent benchmark.
3. **[Uncited]** Claim 7: locate a source for horizon-scaling of hallucinated tool calls, or reframe as our own observation if Results support it. Left as [CITE] — not invented.
4. **[Number]** Claim 6: confirm 23% against halluc2022 p.X; resolve [VALUE — verify].
5. **[Novelty]** Claim 3: either show the comparison that establishes "first," or soften to "to our knowledge."
## Gaps in the library
- No source on horizon-scaling of hallucinated tool calls (Claim 7) — sources/ has nothing on this.End with the calibrated gap count: (supported / overstated / uncited). Never report a section "clean" while [CITE] or [VALUE — verify] markers remain — surface them.
Pairs With
citation-gate.sh(PostToolUse) — proves cite keys resolve; run it first so claim-check is not chasing dangling keys. claim-check is the deeper, source-reading layer it cannot reach.fact-checkeragent — delegate Phase 4 source-reading for a long section: dispatch it to verify a batch of cited claims againstsources/, then fold its findings into the table.block-fabrication.sh(PreToolUse) — if you try to "fix" an uncited claim by writing a stub.bibentry, this hook blocks it. That is the system working: flag[CITE], do not fabricate./citation-audit— run after claim-check if you touched the bibliography, for the structural.bibhealth pass.
Common Rationalizations
| Rationalization | Reality |
| "It's obviously true, no cite needed" | Obvious-to-you is not common-knowledge-for-this-audience. Cite it or it's UNSUPPORTED. |
| "I'll find a citation for this later" | Drafting around a citation you intend to find later is how fabrications enter. Flag [CITE] now. |
| "The source basically says this" | "Basically" is where overclaim lives. The verb and quantifier must match, not the gist. |
| "It's my result, so I can state it strongly" | Your tested harness does not license "in general." Calibrate the quantifier to what you measured. |
| "Reviewer won't check this one" | Reviewer 2 checks exactly the sentence you hoped they'd skip. Assume every claim is read adversarially. |
Notes
- This skill never writes a citation. Its only sourcing output is
[CITE]/[VALUE — verify]placeholders — honest flags, per the cardinal rule inCLAUDE.md. - Verb/quantifier calibration is judgment work — run it on the Reasoner model (see
CLAUDE.md → Model Selection). - Log a recurring overclaim pattern ("you keep using causal verbs in the discussion") to
tasks/reviews/so it becomes a rule, not a one-off fix.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

