agentleFS
nino-chaveznino-chavez/agentic-ways-of-working/skills/evidence-audit/SKILL.md

evidence-audit

Skill2 starsChanged 59 days agoMITView on GitHub

Adversarial provenance audit of a draft deliverable before it ships — re-derive load-bearing claims at their sources, downgrade what fails. Use before publishing any client- or stakeholder-facing artifact containing numbers, ratings, prices, or "verified" language; or when the operator says "audit this", "check the claims", or "is this actually verified?"

6 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

1 rule. 0 are common, 1 is distinctive, and 0 are very specific to this project.

  • 0

    Common

    Most of the 5 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Do not fabricate a fetch you didn't make.

    line 33

“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Evidence Audit

Adversarial audit pass for outbound deliverables. The rule being enforced (working-style.md § Audit discipline): an artifact's claim to have been verified is not verification — and that applies to your own drafts most of all. Search-tool snippets are hearsay; a "verified" label you attached without fetching the source is self-attestation.

Procedure

Honor the request's scope throughout this pass. For an audit-only request, report the exact problem, its consequence, and the smallest remedy; do not rewrite the artifact. Apply corrections when the existing task authorizes editing, without asking again for authorization already given.

  1. Extract the load-bearing claims. Scan the draft for: prices/fees, ratings and review counts, statistics ("1 in 3...", "N% of..."), superlatives tied to numbers, and every instance of "verified / confirmed / checked directly / pulled directly." Load-bearing = a claim whose failure would change the recommendation or embarrass the sender. List them with locations.
  2. Trace each to its actual provenance — from the session transcript, not from memory. For each claim, answer: was the source page fetched in-session (WebFetch/browse-tool), or did this number arrive in a search-result summary? Who owns the domain it came from? Vendor self-comparisons and competitor "analyses" are marketing, not evidence — a competitor's stat about its rival is the same failure as the vendor's stat about itself.
  3. Re-derive what carries "verified" language. Fetch the source directly. Three outcomes:
    • Confirms → keep the label, note the fetch.
    • Contradicts → fix the claim, not the label.
    • Source blocks fetching (G2, Trustpilot 403s) → the claim cannot be labeled verified by anyone; downgrade to "search-corroborated," say the source blocks direct reads, and check whether the recommendation survives without the number. If it doesn't, the recommendation needs different footing.
  4. Check the deliverable's own confidence taxonomy against itself. If the doc promises "green = read at the source," every green tag must trace to a fetch in this session. A confidence system that lies on one row is worse than no confidence system.
  5. Hunt the cost/claim gaps. What does the draft imply is free, complete, or settled that isn't priced or checked? (Origin case: a "$0 to start" claim while the plan's only real recurring cost — video hosting — went unpriced.)
  6. Report findings and authorized corrections. Lead with what the evidence changes and why. In an authorized editing task, apply the corrections and retain conclusions only if the evidence still supports them. In an audit-only task, identify the required corrections and any conclusions that no longer hold.

Notes

  • forge-signal has a deterministic version of steps 1–2 as a pipeline role: src/pipeline/roles/fact-auditor.ts (claim extraction + provenance ledger). This skill is the full pass including re-derivation, which needs live fetches.
  • For client-facing artifact design provenance (palette/type grounded in the client's real brand), see forge-signal/scripts/harvest-brand.mjs.
  • Do not fabricate a fetch you didn't make. "I could not verify X" in the report is a valid, useful finding.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.