agentleFS
dilulloriccardo-archdilulloriccardo-arch/rigor-skills/skills/primary-source-gate/SKILL.md

primary-source-gate

Skill0 starsChanged 2 months agoMITView on GitHub

Invoke before any decision that rests on a number or factual claim — market sizes, earnings figures, pricing, API limits, launch dates, competitor stats. Blocks the decision until every load-bearing claim has a primary source fetched during THIS session.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 5 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    No load-bearing claim enters a decision without a primary source fetched today.

    line 7
  2. Distinctive:

    Before any go/no-go recommendation that cites numbers (revenue potential, market size, user counts, fees, rate limits, deadlines).

    line 13
  3. Distinctive:

    Before writing a research summary, business case, or feasibility report the user will act on.

    line 14
  4. Distinctive:

    When reviewing someone else's research: run every load-bearing claim in it through this gate.

    line 16
  5. Distinctive:

    Extract the claim list: Before concluding anything, write out every load-bearing claim as a one-line statement with its number.

    line 26
  6. Very specific:

    Record URL + fetch date next to the claim: Every load-bearing number in your output carries an inline citation: (source: <URL>, fetched …

    line 35

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Primary Source Gate

No load-bearing claim enters a decision without a primary source fetched today. Not remembered. Not paraphrased from a blog. Fetched, during this session, with the URL and fetch date recorded.

A "load-bearing claim" is any statement that, if false, would change the decision. "The platform pays 80% rev-share" is load-bearing if you are deciding whether to build for that platform. "The platform was founded in 2019" usually is not.

When to use

  • Before any go/no-go recommendation that cites numbers (revenue potential, market size, user counts, fees, rate limits, deadlines).
  • Before writing a research summary, business case, or feasibility report the user will act on.
  • Whenever the agent is about to write a sentence of the form "X earns/costs/supports/allows Y" and Y came from anywhere other than a URL fetched in this session.
  • When reviewing someone else's research: run every load-bearing claim in it through this gate.

When NOT to use

  • Pure code tasks with no external factual claims.
  • Claims that are decorative context, not decision inputs. Do not gate every sentence — gate the ones the decision stands on. If unsure whether a claim is load-bearing, ask: "if this number were half or double, would the recommendation change?" If yes, gate it.
  • Mathematical derivations from already-gated inputs (but gate the inputs).

Procedure

  1. Extract the claim list. Before concluding anything, write out every load-bearing claim as a one-line statement with its number. Example: "Median item on marketplace M earns $24/month." If you cannot enumerate the claims your conclusion depends on, you are not ready to conclude.
  2. Classify each claim's current source tier:
    • Tier 1 (primary): the platform's own API, its live storefront/listing pages, official documentation, an official changelog, a regulatory filing, raw data you can query. This is the only tier that passes the gate.
    • Tier 2 (secondary): journalism, analyst posts, community threads, GitHub READMEs of third parties. These are hypotheses — useful for generating claims to verify, never for verifying them.
    • Tier 3 (adversarial): marketing pages, blogs, and case studies published by the venue that profits when you believe the claim. A platform's blog post about how much its sellers earn is written to recruit sellers, not to inform them. Treat every number in it as a promotional upper bound until the venue's own visible data confirms it.
  3. Fetch the primary source now. During this session. Web-fetch the API endpoint, the doc page, the storefront listing. Do not substitute a cached memory of having read it before — platforms change pricing, limits, and terms without notice, and your training data has a cutoff.
  4. Record URL + fetch date next to the claim. Every load-bearing number in your output carries an inline citation: (source: <URL>, fetched <YYYY-MM-DD>). A claim without this annotation is unverified by definition, and your report must label it as such.
  5. Cross-check adversarial claims against the venue's own visible data. This is the step that catches the most expensive errors. If the venue's marketing says "top performers earn $X", go to the venue's own public storefront, sort by the closest available popularity metric, and estimate what the #1 visible item actually grosses from listed price × visible usage counts. If marketing and storefront disagree, the storefront wins — it is closer to ground truth and harder to inflate.
  6. Apply the staleness rule. A primary source is only primary for the date it describes. Pricing pages, rev-share terms, and rate limits are valid as of the fetch date only. Statistics with an explicit as-of date older than ~6 months must be flagged: "figure is from <date>; current value unverified." For fast-moving venues (AI platforms, marketplaces), 6 months is generous — prefer 90 days.
  7. Gate the output. The final report gets a "Source ledger" section (template below). Any decision-driving claim that could not be tied to a Tier-1 source fetched today is either (a) removed from the argument, or (b) explicitly listed under "Unverified assumptions" with what would change if it were false. Silence is not an option.

Common failure modes

1. Believing the venue's marketing over the venue's own storefront

The trap: A platform's blog claimed "top items in this category earn $500–3,000/month." A research pass took that range as the planning baseline. A later audit opened the platform's own public storefront, sorted the category by usage, and computed that the #1 item was grossing roughly $320/month — below the bottom of the marketed range. Every downstream revenue projection had been anchored 2–10x too high.

Wrong way: "The platform itself says $500–3,000, so it's a primary source." No — the platform's marketing is Tier 3. The platform's storefront data is Tier 1. Same organization, opposite reliability.

Right way: When the entity making the claim profits from your belief in it, find the same entity's operational data (listings, counts, prices, rankings) and recompute the number yourself. If you cannot recompute it, report the marketing figure as "venue-claimed, unverifiable" and never use it as a planning input.

2. Old numbers wearing today's date

The trap: A comparison of platform fee structures cited a rev-share percentage that had been changed two years earlier. The secondary article quoting it was well-written and undated its own sources. The analysis was internally consistent and entirely wrong.

Wrong way: Trusting any number that arrives without an as-of date, or assuming an article's publication date is the data's as-of date.

Right way: Fetch the venue's current terms page today. If the claim matters and the primary page doesn't state an effective date, note "current as of fetch date <today>" — that is the strongest statement you can honestly make. When a secondary source and today's primary source disagree, the discrepancy itself is information: the terms changed, and anything else citing the old figure is also stale.

3. Citing memory as if it were a fetch

The trap: The agent "knows" a platform's rate limit, minimum payout, or category list from training data and states it confidently with no fetch. Training data is a Tier-2 source with an unknown as-of date — the worst possible combination for a load-bearing claim.

Wrong way: "The API allows 100 requests/minute" written from recall, decision proceeds, integration is designed around a limit that was halved last quarter.

Right way: Recall generates the hypothesis and the URL to check — that is its only legitimate role in this gate. The claim enters the report only after the fetch. If the fetch fails (page gone, auth-walled), the claim is unverified and says so.

4. Laundering a secondary source through repetition

The trap: Three blogs all state the same statistic, which feels like corroboration. All three trace to one original post (often itself Tier 3). Citation count is not verification — it is one source wearing three hats.

Right way: Trace the number upstream until you hit either a Tier-1 source or a dead end. A dead end means the number is unverified regardless of how many pages repeat it.

Source ledger template

Include this table in any research output. One row per load-bearing claim.

## Source ledger

| # | Claim | Value | Tier | Source URL | Fetched | Cross-check |
|---|-------|-------|------|------------|---------|-------------|
| 1 | Marketplace median item revenue | $24/mo | 1 | <storefront API url> | 2026-08-25 | computed from 430 listings |
| 2 | Rev-share percentage | 80% | 1 | <terms page url> | 2026-08-25 | terms page, no effective date shown |
| 3 | "Top sellers earn $500-3k/mo" | REJECTED | 3 | <venue blog url> | 2026-08-25 | storefront #1 grosses ~$320/mo — marketing contradicted by venue's own data |

## Unverified assumptions
- <claim>: no Tier-1 source found; if false, <what changes>.

Checklist

  • Every load-bearing claim enumerated before concluding
  • Each claim classified Tier 1 / 2 / 3
  • Tier-1 source fetched during THIS session for every claim used in the argument
  • URL + fetch date recorded inline and in the source ledger
  • Every venue-marketing number cross-checked against the venue's own storefront/API data
  • Every statistic carries an as-of date; anything older than 90 days on a fast-moving venue is flagged stale
  • Repeated citations traced upstream to a single origin before counting as corroboration
  • Unverified claims either removed or listed under "Unverified assumptions" with impact-if-false
  • Zero numbers in the output that exist only in the agent's memory

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.