agentleFS
prime-radiant-incprime-radiant-inc/greenfield/skills/provenance-methodology/SKILL.md

provenance-methodology

Skill284 starsChanged 6 months agoApache-2.0View on GitHub

Cross-cutting provenance discipline. Citation format, confidence levels, source hierarchy, session capture. Loaded by every analysis agent.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 4 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every behavioral claim you write MUST have a citation.

    line 8
  2. Distinctive:

    Cite as you go: Do NOT batch citations at the end of your analysis.

    line 14
  3. Very specific:

    If a single claim is supported by multiple independent sources, use a single citation with corroborated_by listing the additional sources.

    line 44
  4. Distinctive:

    If a paragraph contains multiple claims, each claim gets its own citation.

    line 45
  5. Very specific:

    When escalating, update the <!-- cite: --> comment with the new confidence level and add the confirming source to corroborated_by.

    line 142
  6. Distinctive:

    After each subagent completes, the orchestrator:

    line 193

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Provenance Methodology

Every behavioral claim you write MUST have a citation. No exceptions.

The Citation Rule

A "behavioral claim" is any assertion about what the target system does, how it responds, what data it accepts or produces, what errors it raises, what limits it enforces, or how it transitions between states.

Cite as you go. Do NOT batch citations at the end of your analysis. Every time you write a behavioral claim, the very next thing you write is the citation.

Citation Format

Inline HTML comment, immediately after the claim:

- Sessions expire after 30 minutes of inactivity
  <!-- cite: source=official-docs, ref=https://docs.example.com/sessions#timeout, confidence=confirmed, agent=doc-researcher, corroborated_by=runtime-observation -->

Required Fields

FieldTypeDescription
sourceenumThe type of source (see Source Types below)
refstringSpecific location: URL, workspace/ file path with optional :line, or session timestamp
confidenceenumconfirmed, inferred, or assumed (see Confidence Levels below)
agentstringYour agent name

Optional Fields

FieldTypeDescription
corroborated_bycomma-separated listOther source types that independently confirm this claim
sessionstringAgent ID for session log in workspace/provenance/sessions/

Placement Rules

  • The <!-- cite: --> comment MUST appear on the line immediately following the claim it supports, or on the same line after the claim text.
  • If a single claim is supported by multiple independent sources, use a single citation with corroborated_by listing the additional sources.
  • If a paragraph contains multiple claims, each claim gets its own citation. Break compound sentences into separate cited items.
  • Block-level claims (tables, code blocks, decision trees) place the citation comment immediately after the closing block.

Source Types (Strongest to Weakest)

RankSource TypeWhen to UseOrigin
1official-docsPublished documentation, README, man pages, API references, changelogsPUBLIC
2public-apiObserved behavior of public API endpoints, CLI commandsPUBLIC
3sdk-analysisPublished SDK, client library, or plugin source codePUBLIC
4community-knowledgeStack Overflow, blog posts, conference talks, third-party tutorialsPUBLIC
5runtime-observationBehavior observed by running the product in a containerRAW
6source-codeProprietary source code, bundles, minified JSRAW
7binary-analysisDisassembly, decompilation, binary instrumentationRAW
8inferredReasoning, convention, analogy. No direct observation.N/A

Agent Source Type Guide

AgentPrimary Source Type
doc-researcherofficial-docs or community-knowledge
sdk-analyzer, integration-test-minersdk-analysis
bundle-splitter, chunk-analyzer, function-analyzer, targeted-extractorsource-code
cli-explorer, web-ui-explorer, behavior-observer, ux-documenterruntime-observation
binary-surveyor, binary-deep-analyzerbinary-analysis
Layer 2 synthesis agentsCite upstream agent output files
Layer 3 documentation agentsCite all supporting evidence from any source

Confidence Levels

confirmed

Two or more independent sources agree, OR a single runtime observation with reproducible steps.

Use when:

  • Official docs state X AND source code confirms X
  • Runtime observation shows X AND SDK client handles X
  • Two independent community sources describe X consistently
  • A single reproducible runtime observation (documented input, steps, output)

inferred

One authoritative source, no contradictions.

Use when:

  • Official docs state X but no second source confirms it
  • Source code clearly implements X but no docs mention it
  • A single well-regarded community source describes X, consistent with known behaviors

assumed

Convention, pattern matching, or reasoning. No direct source.

Use when:

  • Following a common convention (e.g., "REST API probably returns JSON") but no source confirms
  • A pattern in one module is assumed to apply in another
  • Evidence is partial and the claim is the most plausible interpretation
  • Filling a spec gap where behavior must exist for the system to function

How to Determine Confidence

digraph confidence_determination {
    rankdir=TB;

    "Determine confidence level" [shape=doublecircle];
    "How many independent sources?" [shape=diamond];
    "Is the source authoritative?" [shape=diamond];
    "Any contradicting evidence?" [shape=diamond];
    "Use confirmed" [shape=box];
    "Use inferred" [shape=box];
    "Use assumed" [shape=box];
    "STOP: Record the contradiction" [shape=octagon, style=filled, fillcolor=red, fontcolor=white];

    "Determine confidence level" -> "How many independent sources?";
    "How many independent sources?" -> "Any contradicting evidence?" [label="2 or more"];
    "How many independent sources?" -> "Is the source authoritative?" [label="1"];
    "How many independent sources?" -> "Use assumed" [label="0"];
    "Is the source authoritative?" -> "Use inferred" [label="yes"];
    "Is the source authoritative?" -> "Use assumed" [label="no"];
    "Any contradicting evidence?" -> "STOP: Record the contradiction" [label="yes"];
    "Any contradicting evidence?" -> "Use confirmed" [label="no"];
}

Authoritative sources: official docs, source code, runtime observation, specific community content. Not authoritative: your own reasoning (always assumed unless corroborated).

Confidence Escalation

Confidence can be upgraded but never downgraded without new contradicting evidence.

FromToTrigger
assumedinferredA single authoritative source confirms the claim
assumedconfirmedTwo independent sources found, OR reproducible runtime observation
inferredconfirmedA second independent source found, OR reproducible runtime observation

When escalating, update the <!-- cite: --> comment with the new confidence level and add the confirming source to corroborated_by.

Handling Contradictions

digraph contradiction_handling {
    rankdir=TB;

    "New source contradicts existing claim" [shape=ellipse];
    "Record both claims with citations" [shape=box];
    "Does runtime observation contradict docs?" [shape=diamond];
    "Does source code contradict docs?" [shape=diamond];
    "Higher-ranked source wins" [shape=box];
    "Runtime observation wins (docs may be stale)" [shape=box];
    "Source code wins for implementation details" [shape=box];
    "Preserve all conflicts in citations for audit" [shape=box];

    "New source contradicts existing claim" -> "Record both claims with citations";
    "Record both claims with citations" -> "Does runtime observation contradict docs?";
    "Does runtime observation contradict docs?" -> "Runtime observation wins (docs may be stale)" [label="yes"];
    "Does runtime observation contradict docs?" -> "Does source code contradict docs?" [label="no"];
    "Does source code contradict docs?" -> "Source code wins for implementation details" [label="yes"];
    "Does source code contradict docs?" -> "Higher-ranked source wins" [label="no"];
    "Higher-ranked source wins" -> "Preserve all conflicts in citations for audit";
    "Runtime observation wins (docs may be stale)" -> "Preserve all conflicts in citations for audit";
    "Source code wins for implementation details" -> "Preserve all conflicts in citations for audit";
}

Common Mistakes

Citing everything as "confirmed" because you feel confident. confirmed requires 2+ independent sources. Your confidence in your own analysis is not a second source. One source = inferred. Period.

Using source=inferred for everything you cannot directly cite. source=inferred means the claim was derived by reasoning. If you read it in docs, source=official-docs. If you saw it in code, source=source-code. Source type = where the evidence came from. Confidence = how sure you are.

Omitting the ref field or using vague references. "ref=docs" is useless. "ref=source code" is useless. Use specific references:

  • ref=https://docs.example.com/auth#tokens (specific URL)
  • ref=workspace/raw/source/analysis/chunk-023.md:45 (file:line)
  • ref=provenance/sessions/cli-explorer-abc123.jsonl:2026-03-01T10:30:00Z (session:timestamp)

Not recording assumed claims. Unrecorded assumptions look like confirmed facts to downstream agents and the implementer. Always cite assumed claims with confidence=assumed. List them in an Assumptions section. This is honesty, not weakness.

Batching citations at the end. You forget sources. You guess at confidence. Citations become decoration instead of evidence. Cite as you go.

Session Capture

After each subagent completes, the orchestrator:

  1. Copies the session JSONL to workspace/provenance/sessions/{agent-name}-{agent-id}.jsonl
  2. Records the agent run in workspace.json
  3. Appends to the audit log at workspace/provenance/audit-log.md
  4. Creates a git commit containing all output files, session log, and updated metadata

The session log contains every tool call, search result, and file reading that the agent performed. It is the deepest layer of provenance evidence.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.