provenance-methodology
Skill284 starsChanged 6 months agoApache-2.0View on GitHub
Cross-cutting provenance discipline. Citation format, confidence levels, source hierarchy, session capture. Loaded by every analysis agent.
In this repository21 other files it gives its agents.
- analysis-pipelineSkill
- autonomous-discoverySkill
- behavioral-spec-writingSkill
- binary-analysisSkill
- community-intelligenceSkill
- container-executionSkill
- contract-detectionSkill
- doc-researchSkill
- ecosystem-analysisSkill
- fidelity-validationSkill
- git-archaeologySkill
- incremental-analysisSkill
- multi-source-synthesisSkill
- runtime-observationSkill
- second-pass-reviewSkill
- source-analysisSkill
- source-completenessSkill
- spec-sanitizationSkill
- test-suite-analysisSkill
- validation-methodologySkill
- visual-explorationSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 4 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Every behavioral claim you write MUST have a citation.
- Distinctive: line 14
Cite as you go: Do NOT batch citations at the end of your analysis.
- Very specific: line 44
If a single claim is supported by multiple independent sources, use a single citation with corroborated_by listing the additional sources.
- Distinctive: line 45
If a paragraph contains multiple claims, each claim gets its own citation.
- Very specific: line 142
When escalating, update the <!-- cite: --> comment with the new confidence level and add the confirming source to corroborated_by.
- Distinctive: line 193
After each subagent completes, the orchestrator:
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Provenance Methodology
Every behavioral claim you write MUST have a citation. No exceptions.
The Citation Rule
A "behavioral claim" is any assertion about what the target system does, how it responds, what data it accepts or produces, what errors it raises, what limits it enforces, or how it transitions between states.
Cite as you go. Do NOT batch citations at the end of your analysis. Every time you write a behavioral claim, the very next thing you write is the citation.
Citation Format
Inline HTML comment, immediately after the claim:
- Sessions expire after 30 minutes of inactivity
<!-- cite: source=official-docs, ref=https://docs.example.com/sessions#timeout, confidence=confirmed, agent=doc-researcher, corroborated_by=runtime-observation -->Required Fields
| Field | Type | Description |
source | enum | The type of source (see Source Types below) |
ref | string | Specific location: URL, workspace/ file path with optional :line, or session timestamp |
confidence | enum | confirmed, inferred, or assumed (see Confidence Levels below) |
agent | string | Your agent name |
Optional Fields
| Field | Type | Description |
corroborated_by | comma-separated list | Other source types that independently confirm this claim |
session | string | Agent ID for session log in workspace/provenance/sessions/ |
Placement Rules
- The
<!-- cite: -->comment MUST appear on the line immediately following the claim it supports, or on the same line after the claim text. - If a single claim is supported by multiple independent sources, use a single citation with
corroborated_bylisting the additional sources. - If a paragraph contains multiple claims, each claim gets its own citation. Break compound sentences into separate cited items.
- Block-level claims (tables, code blocks, decision trees) place the citation comment immediately after the closing block.
Source Types (Strongest to Weakest)
| Rank | Source Type | When to Use | Origin |
| 1 | official-docs | Published documentation, README, man pages, API references, changelogs | PUBLIC |
| 2 | public-api | Observed behavior of public API endpoints, CLI commands | PUBLIC |
| 3 | sdk-analysis | Published SDK, client library, or plugin source code | PUBLIC |
| 4 | community-knowledge | Stack Overflow, blog posts, conference talks, third-party tutorials | PUBLIC |
| 5 | runtime-observation | Behavior observed by running the product in a container | RAW |
| 6 | source-code | Proprietary source code, bundles, minified JS | RAW |
| 7 | binary-analysis | Disassembly, decompilation, binary instrumentation | RAW |
| 8 | inferred | Reasoning, convention, analogy. No direct observation. | N/A |
Agent Source Type Guide
| Agent | Primary Source Type |
doc-researcher | official-docs or community-knowledge |
sdk-analyzer, integration-test-miner | sdk-analysis |
bundle-splitter, chunk-analyzer, function-analyzer, targeted-extractor | source-code |
cli-explorer, web-ui-explorer, behavior-observer, ux-documenter | runtime-observation |
binary-surveyor, binary-deep-analyzer | binary-analysis |
| Layer 2 synthesis agents | Cite upstream agent output files |
| Layer 3 documentation agents | Cite all supporting evidence from any source |
Confidence Levels
confirmed
Two or more independent sources agree, OR a single runtime observation with reproducible steps.
Use when:
- Official docs state X AND source code confirms X
- Runtime observation shows X AND SDK client handles X
- Two independent community sources describe X consistently
- A single reproducible runtime observation (documented input, steps, output)
inferred
One authoritative source, no contradictions.
Use when:
- Official docs state X but no second source confirms it
- Source code clearly implements X but no docs mention it
- A single well-regarded community source describes X, consistent with known behaviors
assumed
Convention, pattern matching, or reasoning. No direct source.
Use when:
- Following a common convention (e.g., "REST API probably returns JSON") but no source confirms
- A pattern in one module is assumed to apply in another
- Evidence is partial and the claim is the most plausible interpretation
- Filling a spec gap where behavior must exist for the system to function
How to Determine Confidence
digraph confidence_determination {
rankdir=TB;
"Determine confidence level" [shape=doublecircle];
"How many independent sources?" [shape=diamond];
"Is the source authoritative?" [shape=diamond];
"Any contradicting evidence?" [shape=diamond];
"Use confirmed" [shape=box];
"Use inferred" [shape=box];
"Use assumed" [shape=box];
"STOP: Record the contradiction" [shape=octagon, style=filled, fillcolor=red, fontcolor=white];
"Determine confidence level" -> "How many independent sources?";
"How many independent sources?" -> "Any contradicting evidence?" [label="2 or more"];
"How many independent sources?" -> "Is the source authoritative?" [label="1"];
"How many independent sources?" -> "Use assumed" [label="0"];
"Is the source authoritative?" -> "Use inferred" [label="yes"];
"Is the source authoritative?" -> "Use assumed" [label="no"];
"Any contradicting evidence?" -> "STOP: Record the contradiction" [label="yes"];
"Any contradicting evidence?" -> "Use confirmed" [label="no"];
}Authoritative sources: official docs, source code, runtime observation, specific community content. Not authoritative: your own reasoning (always assumed unless corroborated).
Confidence Escalation
Confidence can be upgraded but never downgraded without new contradicting evidence.
| From | To | Trigger |
assumed | inferred | A single authoritative source confirms the claim |
assumed | confirmed | Two independent sources found, OR reproducible runtime observation |
inferred | confirmed | A second independent source found, OR reproducible runtime observation |
When escalating, update the <!-- cite: --> comment with the new confidence level and add the confirming source to corroborated_by.
Handling Contradictions
digraph contradiction_handling {
rankdir=TB;
"New source contradicts existing claim" [shape=ellipse];
"Record both claims with citations" [shape=box];
"Does runtime observation contradict docs?" [shape=diamond];
"Does source code contradict docs?" [shape=diamond];
"Higher-ranked source wins" [shape=box];
"Runtime observation wins (docs may be stale)" [shape=box];
"Source code wins for implementation details" [shape=box];
"Preserve all conflicts in citations for audit" [shape=box];
"New source contradicts existing claim" -> "Record both claims with citations";
"Record both claims with citations" -> "Does runtime observation contradict docs?";
"Does runtime observation contradict docs?" -> "Runtime observation wins (docs may be stale)" [label="yes"];
"Does runtime observation contradict docs?" -> "Does source code contradict docs?" [label="no"];
"Does source code contradict docs?" -> "Source code wins for implementation details" [label="yes"];
"Does source code contradict docs?" -> "Higher-ranked source wins" [label="no"];
"Higher-ranked source wins" -> "Preserve all conflicts in citations for audit";
"Runtime observation wins (docs may be stale)" -> "Preserve all conflicts in citations for audit";
"Source code wins for implementation details" -> "Preserve all conflicts in citations for audit";
}Common Mistakes
Citing everything as "confirmed" because you feel confident.
confirmed requires 2+ independent sources. Your confidence in your own analysis is not a second source. One source = inferred. Period.
Using source=inferred for everything you cannot directly cite.
source=inferred means the claim was derived by reasoning. If you read it in docs, source=official-docs. If you saw it in code, source=source-code. Source type = where the evidence came from. Confidence = how sure you are.
Omitting the ref field or using vague references.
"ref=docs" is useless. "ref=source code" is useless. Use specific references:
ref=https://docs.example.com/auth#tokens(specific URL)ref=workspace/raw/source/analysis/chunk-023.md:45(file:line)ref=provenance/sessions/cli-explorer-abc123.jsonl:2026-03-01T10:30:00Z(session:timestamp)
Not recording assumed claims.
Unrecorded assumptions look like confirmed facts to downstream agents and the implementer. Always cite assumed claims with confidence=assumed. List them in an Assumptions section. This is honesty, not weakness.
Batching citations at the end. You forget sources. You guess at confidence. Citations become decoration instead of evidence. Cite as you go.
Session Capture
After each subagent completes, the orchestrator:
- Copies the session JSONL to
workspace/provenance/sessions/{agent-name}-{agent-id}.jsonl - Records the agent run in
workspace.json - Appends to the audit log at
workspace/provenance/audit-log.md - Creates a git commit containing all output files, session log, and updated metadata
The session log contains every tool call, search result, and file reading that the agent performed. It is the deepest layer of provenance evidence.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

