source-verify
Skill0 starsChanged 3 months agoMITView on GitHub
Use when evidence records exist with verification_status pending and need to be checked before synthesis — either because the user asks to "verify these sources", "check if this quote actually supports the claim", "audit this evidence", or because the orchestrator has just finished a collection pass and is handing off to verification before synthesis. This is the mandatory station between any source skill and the synthesizer — no source skill's evidence reaches findings.jsonl without passing through here first. Do NOT use this to collect new evidence (that's the individual source skills) or to compute finding status (that's the deterministic validator, scripts/validate_run.py — this skill only resolves evidence-level verification_status).
1 other project uses a version of this.
Adopted by
Most recently taken up by
- Konata9/writing-toolkittheir own version, first seen Oct 16, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- Konata9/writing-toolkit★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository10 other files it gives its agents.
- academic-researchSkill
- blog-researchSkill
- github-researchSkill
- news-researchSkill
- official-source-researchSkill
- reddit-researchSkill
- research-intelligenceSkill
- research-plannerSkill
- social-researchSkill
- web-researchSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 3 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 16
Every pending record gets both checks, in order.
- Very specific: line 29
Not found in either → rejected (mechanical: not-found).
- Very specific: line 32
archive-failed:is not a URL: When archive_url starts with archive-failed:, there is no archive to try — skip straight to the outcome the … - Very specific: line 36
Read the passage surrounding the quote — not just the quote in isolation.
- Distinctive: line 47
When check (b) finds partial support:
- Very specific: line 67
Does not decide finding status (confirmed/probable/contested/insufficient) — that ladder is computed only by scripts/validate_run.py, …
- Distinctive: line 68
Does not synthesize, compare across records, or write findings.
- Distinctive: line 69
Does not re-collect evidence — if a claim needs different/better evidence, that goes back to the relevant source skill, not to this station.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Source Verify
Overview
The mandatory checkpoint every evidence record passes through before it can be cited by any finding (constitution rule 6). No source skill verifies its own output — the actor that collects evidence never grades it. This skill's only job: turn verification_status: pending into verified, rejected, or unreachable, honestly, one record at a time.
Layer rule: this is where network-dependent verification happens (constitution rule 8) — archive liveness checks, re-fetching URLs, confirming a page still says what the record claims. scripts/validate_run.py never touches the network; everything network-dependent about "is this evidence real" happens here, once, before the validator ever runs.
The two checks (constitution rule 6)
Every pending record gets both checks, in order. Both must pass for verified.
Check (a) — Mechanical: does the quote exist?
Fetch source.url. Search for the evidence.quote text, whitespace-normalized (collapse multiple spaces/newlines, ignore leading/trailing whitespace) but otherwise exact — no fuzzy matching, no "close enough."
Multi-part quotes. A quote trimmed with [...] is split on that marker into parts. Every part must appear at the source, in the same order, with no more than ~2000 characters between consecutive parts. Without the order and distance constraints, "exact quote" means something different in every implementation, and a quote could be assembled from fragments scattered across an entire page.
Fall through to the archive on either failure mode, not just one:
- Live URL loads and contains the quote → check (a) passes,
checked_via: live. - Live URL loads but the quote is not there → try the archive anyway. A page edited after collection is the normal case here, not an exception: the evidence was real when collected, and the snapshot is what proves it. Found in the archive → passes,
checked_via: archive, andlimitationsrecords that the live page has since changed. Treating a content-drifted page as a rejection would discard true evidence and quietly punish sources that update. - Live URL fails to load → try the archive. Found → passes,
checked_via: archive. - Not found in either → rejected (
mechanical: not-found). Do not proceed to check (b). - Neither loads at all → unreachable (
mechanical: unreachable), notrejected— an availability failure is not evidence the claim is false.
archive-failed: is not a URL. When archive_url starts with archive-failed:, there is no archive to try — skip straight to the outcome the live check produced. Never issue a fetch against that marker.
Check (b) — Semantic: does the quote, in context, support the claim?
Read the passage surrounding the quote — not just the quote in isolation. Ask: standing in its actual context, does this passage assert what claim says, at the scope claim states (same version, region, date range, magnitude)?
How much context. "Surrounding" needs a definition or the same record verifies differently on different days. Start with the quote's own paragraph, expand to the paragraph before and after, and expand once more to the enclosing section only if the claim is still ambiguous. Per source type: HTML → paragraph, then section; PDF → paragraph, then page; GitHub code → the enclosing function or block, then the file's surrounding context; GitHub issues and community threads → the parent comment plus the thread's opening post, since a reply's meaning frequently lives in what it is replying to.
- Full support → check (b) passes →
verified. - No support (context contradicts, or the quote was cherry-picked to imply something the passage doesn't actually say) → rejected.
- Partial support (the passage supports part of the claim but not all of it — a narrower scope, a smaller number, a qualified rather than absolute statement) → not a rejection. Run the shrink procedure below.
- The passage itself attributes the claim to another source ("according to X", "as reported by", wire-service framing, a citation to a paper) → do not verify this record as-is. Route back to engine/origin-tracing.md to resolve the actual root, then re-run both checks against the root's own record. The original record, if kept at all, is marked with
root_originpointing at the resolved root and is not independentlyverifiedas if it originated the claim.
Shrink → re-verify (constitution rule 6, "partial is not a status")
When check (b) finds partial support:
- Rewrite
claimto state exactly what the quote supports — no more. ("Revenue grew 40% globally" → "Revenue grew 40% in the enterprise segment" if that's what the passage actually says.) - Append to
limitations:"claim narrowed from: <original claim text>"— the narrowing is never silent. - Increment
claim_revision. This is what makes the narrowing visible downstream: any finding citing this record must carry a matchingacknowledged_revisionsentry, or validation fails (engine/synthesis-contract.md). Without this, a finding written against the original broad claim would silently outlive the evidence that no longer supports it — the validator cannot compare a finding's sentence to a claim semantically, so the acknowledgment is what forces a human/synthesizer to look again. - Re-run check (b) against the new claim. It should now pass (the claim was built from what the quote supports) — if it somehow still doesn't, treat as
rejectedrather than looping. - Set
verification_status: verifiedon the narrowed record.
This record is never discarded for being narrower than first collected — a correctly-scoped verified record is more valuable than a broad rejected one. Log the narrowing in the run ledger too, so the synthesizer knows which findings need re-examination.
Procedure
- Pull every
pendingrecord from the run'sevidence.jsonl. - For each: run check (a), then check (b) if (a) passed. Apply shrink if partial. Apply the origin-tracing routing if the quote is itself a relay.
- Write the resolved
verification_statusback to the record, together with theverificationblock (mechanical,semantic,checked_via,checked_at). Recording only the final status loses the two things an auditor most wants later: which of the two checks decided it, and whether the quote was confirmed against the live source or an archived snapshot. This station is the only one allowed to write either field after collection. - Never delete a
rejectedorunreachablerecord. They stay inevidence.jsonlas a permanent trail of what was checked and why it didn't hold up —rejectedevidence simply becomes uncitable (engine/synthesis-contract.md: no finding may cite it). - Log a short verification summary per run in the ledger: counts of verified/rejected/unreachable/narrowed, and which records were rerouted to origin-tracing.
What this skill does not do
- Does not decide finding
status(confirmed/probable/contested/insufficient) — that ladder is computed only byscripts/validate_run.py, deterministically, from verified evidence. This skill's output (verification_status) is one of that computation's inputs, nothing more. - Does not synthesize, compare across records, or write findings.
- Does not re-collect evidence — if a claim needs different/better evidence, that goes back to the relevant source skill, not to this station.
Common mistakes
| Mistake | Fix |
Treating "URL is down" as rejected | That's unreachable — a different failure mode with different downstream handling |
| Discarding a partially-supported record | Shrink the claim and re-verify; never discard for being narrower |
Shrinking a claim without bumping claim_revision | The bump is what forces findings to be re-checked against the narrowed claim — skipping it reintroduces the exact overclaim the shrink just fixed |
| Verifying a record whose quote attributes the claim elsewhere | Route to origin-tracing first, verify the resolved root instead |
| Checking the quote against the live URL only, skipping the archive on a fetch failure | Always try the archive before concluding unreachable |
| Rejecting a record because the live page no longer contains the quote | Try the archive first — an edited page is the normal case, and the snapshot is what the evidence was collected against |
Fetching archive-failed:<reason> as if it were a URL | It is a marker, not an address; there is no archive to check |
Letting the collecting source skill mark its own evidence verified | This station is separate from collection precisely so that never happens |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

