agentleFS
ozturkogluozturkoglu/fable-skills/skills/verification-chain/SKILL.md

verification-chain

Skill0 starsChanged 4 months agoMITView on GitHub

A five-step pipeline for turning a research, analysis, or investigative finding into something you can actually trust before it's reported as fact: self-research, a free second opinion, self-verification of that opinion, exactly one top-tier audit pass, then a final report. Trigger on explicit asks like "verify this", "audit this", "double-check this", "can we trust this", "sanity-check this", "is this really true", "fact check this", "research this thoroughly", "check my work", "get a second opinion", "don't just take my word for it", or any request for a reliable/defensible conclusion on a multi-step analysis, investigation, or claim — not on simple lookups, single-file edits, or anything with one obvious correct answer.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 5 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Before the first run in a new project, ask the user two questions and keep the answers in a short local note so this isn't re-asked every …

    line 27
  2. Distinctive:

    Research directly: Use the model already running the session — do not spawn a same-tier subagent for this step.

    line 41
  3. Distinctive:

    Self-verify: Check every claim the second opinion raised against real data — a file read, a re-run computation, a source fetch.

    line 49
  4. Distinctive:

    If the top tier is unavailable, step 4 falls back to the next-highest available tier.

    line 83
  5. Distinctive:

    Use this when the user wants a conclusion they can rely on for a non-trivial analysis.

    line 119

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Verification Chain

Purpose: give the user a conclusion they can actually trust, at the lowest cost that still catches real errors. Built for one recurring failure: a single pass — by any one model, at any tier — misses things a second, independent pass catches. But throwing many expensive parallel passes at the problem doesn't scale the catch-rate the way it scales the bill.

Setup (once per environment)

Before the first run in a new project, ask the user two questions and keep the answers in a short local note so this isn't re-asked every time:

  1. "Do you have a free or already-paid-for second-opinion tool available — a browser-automated chat session, another vendor's CLI, a local model? What's the exact command or process to invoke it?" If none, skip step 2 of the chain below; steps 1, 3 (verify-directly instead), 4, and 5 still work without it.
  2. "Confirm you have access to a top-tier ('frontier') model for the final audit pass, and name a fallback tier for when the top tier is rate-limited, quota-exhausted, or unavailable."

The chain

  1. Research directly. Use the model already running the session — do not spawn a same-tier subagent for this step. It adds cost without adding a genuinely second perspective.
  2. Get a free second opinion. Hand the finding to the configured second-opinion tool, framed either adversarially ("try to refute this") or constructively ("how would you strengthen or test this further"), depending on where the finding stands. Zero marginal cost if the tool is already free or already paid for.
  3. Self-verify. Check every claim the second opinion raised against real data — a file read, a re-run computation, a source fetch. An unverified claim from step 2 does not enter the report. (This step composes with execution-guardrails's verify-before-flag rule: it's that rule, applied specifically to the second opinion's output.)
  4. Exactly one top-tier audit pass. Send the verified package to the top-tier model. Ask it to independently re-check at least one claim itself — not just read the summary — and render the final verdict. If the top tier is unavailable, the configured fallback tier takes the same role (see "Tier fallback").
  5. Report. The session model from step 1 writes and delivers the final report. The audit model's output is a verdict-plus-reasoning input to that report, not the report itself.

Failable check: the delivered report names which tool performed step 2, what step 3 confirmed or rejected from it, and which tier performed step 4. No named provenance for any step means the chain wasn't actually followed.

Rule — one audit pass, not a gate at every stage

Step 4 runs once per finding, at the end. Do not re-insert the top-tier model at multiple checkpoints along the way — in practice this multiplies cost several-fold without a matching gain in what gets caught, because the single end-of-chain pass already has the complete, self-verified picture to work from.

Exception: a finding that gates an irreversible or high-stakes action (shipping code, spending money, anything the user has flagged as high-stakes) can justify an earlier checkpoint too — a deliberate exception, not the default.

Tier fallback

If the top tier is unavailable, step 4 falls back to the next-highest available tier. This is not guaranteed to be equivalent — but in one blind test (see Evidence) the fallback tier reproduced the top tier's verdict and full reasoning structure at roughly two-thirds the token cost of the top-tier pass it stood in for. Treat the fallback as a validated option, not an unproven guess — while still flagging in the report that a fallback tier was used.

Evidence

Measured on a real multi-stage research-verification task: four pipeline variants were run head-to-head with token cost tracked for each. A same-tier self-plus-adversarial-subagent pipeline cost the most and caught nothing the free-second-opinion pipeline didn't also catch. The free-second-opinion-plus-single-top-tier-audit pipeline (this skill's shape) was the cheapest per genuine finding caught. A separate blind re-run — handing an already-adjudicated finding to a fallback-tier model without revealing the original verdict — reproduced the same conclusion and the same gate-by-gate reasoning at about two-thirds the token cost of the original top-tier pass. This is one comparison, not a universal guarantee — worth repeating on your own work before treating it as settled.

Differs from the other skills here

  • model-routing picks a tier for a single task. This skill is a five-step process that uses model-routing's tier ladder internally (steps 1, 2/3, and 4 each route to a different tier) and adds the second-opinion-plus-single-audit-pass structure on top.
  • execution-guardrails is an always-on constraint against reporting unverified claims. This skill's step 3 is that guardrail applied specifically to a second opinion's output — the two compose, they don't overlap.
  • setup-audit audits your Claude Code setup itself. This skill audits a piece of analysis — a different subject entirely.

Use this when the user wants a conclusion they can rely on for a non-trivial analysis. Don't use it for a single-file edit, a direct factual lookup, or anything with one obvious correct answer.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.