Research Methodology
Skill0 starsChanged 4 months agoMITView on GitHub
Disciplined investigation of unknowns in any context — codebases, tools, APIs, incidents, claims. Auto-load when the task is to investigate, research, verify, audit, or answer "why/how/whether" questions; when uncertainty exists about how a system behaves; or when findings will feed a decision. Enforces explicit hypotheses, disconfirming evidence, primary-source verification (code > docs > memory), explicit epistemic status labels (verified / inferred / hypothesis / requires verification), date-stamping of volatile findings, digest-form output, and explicit stopping criteria so research converges into action.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- ralfyishere/rules-with-receiptstheir own version, first seen Jul 8, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- ralfyishere/rules-with-receipts★ 2same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository25 other files it gives its agents.
- Agent EngineeringSkill
- API and Interface DesignSkill
- Architecture AnalysisSkill
- Build and ReleaseSkill
- Campaign PlanningSkill
- Change ControlSkill
- Codebase OnboardingSkill
- Code Review PlaybookSkill
- Configuration ManagementSkill
- Debugging PlaybookSkill
- Dependency ManagementSkill
- Documentation PracticesSkill
- Engineering MinimalismSkill
- Evaluation FrameworksSkill
- Failure ArchaeologySkill
- LLM Integration ReliabilitySkill
- LLM System DesignSkill
- Observability and DiagnosticsSkill
- Performance EngineeringSkill
- Prompt and Context EngineeringSkill
- Proof and AnalysisSkill
- Refactoring PlaybookSkill
- Security Review PlaybookSkill
- System DesignSkill
- Validation And TestingSkill
The file
We split this skill into the rules it gives an agent
7 rules. About 0 are common, 6 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈6
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 37
Write down what you *expect* to find and why.
- Distinctive: line 72
Never present an assumption as a fact.
- Distinctive: line 91
Stop researching and act when ANY of:
- Distinctive: line 98
Do NOT stop while a *blocking* claim is still labeled hypothesis — that is exactly the claim that must be verified or explicitly flagged to …
- Distinctive: line 104
Name the incumbent: Every adoption question is a comparison.
- Distinctive: line 190
Every claim in your output carries one of the four epistemic labels (or an obvious verified citation).
- Very specific: line 200
Assumptions: commit hashes (a1ad63ed, 0a06ae9b, e264a72, b7bbabb, 51b92c9, b011239) and quoted commit messages are point-in-time facts from …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Research Methodology
Purpose
Investigation fails in two ways: presenting assumptions as facts, and researching forever instead of acting. This skill is the runbook for both — every claim gets an explicit epistemic status backed by a primary source, and every investigation has stated stopping criteria. The output is always a digest a decision-maker (human or agent) can act on without re-deriving the work.
When to Use / When NOT to Use
Use when:
- Answering "why does X happen?", "how does Y work?", "is claim Z true?" about any system.
- Investigating an unfamiliar tool, API, dependency, or protocol before building on it.
- Auditing statements someone else made (docs, comments, commit messages, a previous agent's report).
- Producing findings another agent or person will act on — the epistemic labels are the handoff contract.
- Model memory conflicts with observed behavior.
Do NOT use when:
- First contact with a whole codebase (structure, entry points, conventions) → load
codebase-onboarding; return here for specific open questions it surfaces. - Chasing a live defect with a reproducer → load
debugging-playbook(hypothesis discipline overlaps, but that skill owns the reproduce–bisect–fix loop). - Reconstructing how a past failure happened from history → load
failure-archaeology. - Mapping components and dependencies of a system → load
architecture-analysis. - The unknown is "should this code exist at all?" → load
engineering-minimalism. - The answer is already verified and you're stalling — stop researching, act (see step 8).
Core Methodology
1. Write the question and the decision it feeds
One sentence each: "Question: does X do Y?" and "Decision: if yes we do A, if no we do B." If no decision depends on the answer, the research is speculative — stop (and consider engineering-minimalism).
2. Form explicit hypotheses before looking
Write down what you expect to find and why. A hypothesis you didn't write down mutates silently into "what I always thought" after you see the evidence. Multiple competing hypotheses are better than one.
3. Rank your sources: code > docs > memory
- Primary: the code itself, git history, actual command output, actual API responses. These can be quoted.
- Secondary: README/docs/comments — statements of intent, which drift from behavior. Trust only after spot-checking against primary.
- Tertiary: your training memory. Never sufficient alone for any claim that matters; APIs, versions, and flags drift past any knowledge cutoff. Memory generates hypotheses; it never confirms them.
Source decision tree for any claim you are about to write down:
Claim about to be written
├── Did I observe it directly? (read the code, ran the command, saw output)
│ └── YES → label VERIFIED, cite the file/commit/command
├── Does it follow logically from verified facts?
│ └── YES → label INFERRED, state the reasoning chain
├── Do I merely expect it? (docs said so, memory says so, "usually true")
│ ├── Can I check it cheaply now? → CHECK IT, then relabel
│ └── Can't check now → label HYPOTHESIS or REQUIRES VERIFICATION
└── None of the above → do not write the claim4. Seek disconfirming evidence
For each hypothesis, ask: "what would I see if this were FALSE?" — then look for that, specifically. Grep for the counter-example, read the error path not just the happy path, check the commit that removed something, not only the one that added it. Confirmation-only research is how wrong answers arrive with high confidence.
5. Label every claim with epistemic status
Four labels, defined:
- verified — directly observed against a primary source; citation attached (path, commit hash, command + output).
- inferred — logical consequence of verified facts; reasoning stated.
- hypothesis — plausible, believed, unchecked.
- requires verification — needed for the decision but not yet checked; blocking or non-blocking stated.
Never present an assumption as a fact. In prose, the parenthetical form is fine: "(hypothesis — requires verification)". Un-labeled claims default to hypothesis in the reader's mind — so label them yourself, honestly.
6. Date-stamp volatile findings
Anything that can drift — versions, commit tips, published package state, dist-tags, API behavior, config values — gets a stamp: "(as of YYYY-MM-DD)", filled with the actual verification date. Stable findings (an algorithm in a pinned file at a named commit) may cite the commit instead. A dated finding is re-checkable; an undated one is a future lie.
7. Structure findings into digest form
The deliverable is a digest, not a transcript:
- Question + decision it feeds (from step 1)
- Answer in 1–3 sentences, with overall confidence
- Key findings, each with epistemic label + citation + date stamp
- What was ruled out (disconfirmed hypotheses — this is where step 4's work shows)
- Open items — each "requires verification" entry, with the exact command to close it
- Recommended action
8. Know when to stop
Stop researching and act when ANY of:
- The decision from step 1 is now determined either way.
- Remaining unknowns are labeled, non-blocking, and have re-check commands attached.
- The last two search rounds produced no new relevant facts (diminishing returns).
- Cost of further research exceeds the cost of acting and being cheaply wrong (reversible action + labeled uncertainty beats another hour of reading).
Do NOT stop while a blocking claim is still labeled hypothesis — that is exactly the claim that must be verified or explicitly flagged to the decision-maker.
Frontier adoption — evaluating new tools, libraries, models, and techniques
A special case of steps 1–8 with its own gate. When the question is "should we adopt X?" (a new framework, model, library, or technique):
- Name the incumbent. Every adoption question is a comparison. "Adopt X" really means "replace [current approach] with X" — even when the incumbent is "nothing / hand-rolled." If you cannot name what X replaces, return to step 1 of the core method: no decision depends on the answer.
- Demand evidence at the right tier. Marketing pages and benchmark claims are tertiary. A maintained repo with recent commits, an issue tracker you read (open bugs ARE documentation), and a working spike you ran yourself are primary. Never adopt on secondary evidence alone.
- Run a bounded spike. Time-box a minimal integration (hours, not days) against your real use case, not the tool's demo. The spike's output is a digest (step 7) with a recommendation, not production code.
- Price the full cost. Adoption cost = integration + the dependency's transitive surface (
dependency-management) + team/agent learning curve + exit cost if it dies. Compare against the incumbent's known cost, not against zero. - Default to the incumbent on a tie. Novelty is not a tiebreaker; switching costs are real and the incumbent's failure modes are already known (
engineering-minimalismrung 1 applies: is the need real, today?). - Date-stamp the verdict. Frontier facts rot fastest of all — a "not ready" verdict on a fast-moving tool deserves a re-check date, not permanence.
Investigation checklist
- Question and dependent decision written (1 sentence each)
- Hypotheses written down before evidence-gathering
- Every claim traced to a primary source or labeled otherwise
- Disconfirming evidence actively sought for each hypothesis
- All four epistemic labels used honestly (an all-"verified" report is suspicious)
- Volatile findings date-stamped
- Digest produced (answer, findings, ruled-out, open items, action)
- Stopping criterion stated and met
Discovery Commands
Primary-source interrogation, repo-agnostic:
# What actually happened, in order (never trust a summary of history)
git log --oneline -30
git log --oneline --all --graph | head -40
# Verify a specific claim about a change: read the actual diff
git show <commit-hash>
git show <commit-hash> --stat
# When/why did this line come to exist?
git blame -L <start>,<end> <file>
# Find when a string appeared/disappeared anywhere in history
git log -S "search-string" --oneline
# Hunt the counter-example (disconfirming evidence)
grep -rn "the-thing-that-should-not-exist-if-hypothesis-true" .
# Does the claimed file/flag/config actually exist?
find . -name "<claimed-file>" -not -path "*/node_modules/*" -not -path "*/.git/*"
ls -la <claimed-path>
# What does the project SAY it does (secondary source, to be spot-checked)
find . -maxdepth 2 -iname "README*" -o -iname "*.md" | head
# Verify declared dependencies/versions against reality
grep -n "version\|dependencies" package.json pyproject.toml setup.py 2>/dev/nullRule: any claim in your digest that cannot be regenerated by one of these command patterns is not "verified" — relabel it.
Failure Modes & Anti-patterns
| Symptom | Mistake | Correction |
| Confident answer turns out wrong | Memory presented as fact | Memory only generates hypotheses; verify against code (step 3) |
| Docs say X, code does Y, report said X | Secondary source trusted without spot-check | Code > docs; quote the code (step 3) |
| Only supporting evidence in the digest | Confirmation-seeking | Ask "what would I see if false?" and look for it (step 4) |
| Reader can't tell facts from guesses | Missing epistemic labels | Label every claim; unlabeled = hypothesis (step 5) |
| Finding was true in March, acted on in July, broke | No date stamp on volatile fact | Stamp "(as of DATE)"; attach re-check command (step 6) |
| 40-message investigation, no conclusion | No stopping criteria | Decision-linked question + stop conditions upfront (steps 1, 8) |
| Handoff report forces reader to redo the work | Transcript instead of digest | Digest form: answer, findings, ruled-out, open items (step 7) |
| Hypothesis silently became "known" mid-thread | Hypotheses not written before looking | Write expectations down first (step 2) |
| Blocking unknown shipped as a footnote | "Requires verification" not triaged | Mark blocking vs non-blocking; never act past a blocking one (step 8) |
| Research on a question nobody needs answered | No dependent decision | If no decision changes, stop; consider engineering-minimalism (step 1) |
Repository Examples
Case study: ruflo — the claim that drifted (as of 2026-07-04)
~/ruflo — pnpm monorepo publishing three npm package aliases at identical versions with coordinated dist-tags. The docs mark the ruflo alias dist-tag step "EASY TO FORGET" — a written acknowledgment that a claim like "all aliases are published consistently" is volatile and needs per-release re-verification, not memory. The slip actually happened: commit "fix: @claude-flow/browser peer dep, dist-tags, bump to alpha.3" is the primary-source evidence that the documented risk materialized. Lesson applied to method: registry state and dist-tags are tertiary-trust items — verify at the registry, date-stamp the finding, never carry it across sessions unverified.
Case study: ragit — verifying a dependency claim via history (as of 2026-07-04)
~/ragit — local RAG CLI. The claim "numpy must stay <2.0 here" is not folklore: it is verified by commits a1ad63ed "Pin chromadb to 0.4.x" and 0a06ae9b "Pin numpy for chromadb 0.4", pinning chromadb>=0.4.0,<0.5.0 + numpy<2.0 after a transitive break. Re-check command: cd ~/ragit && git show 0a06ae9b. This is the pattern: a pin is a finding with provenance — the hash is the citation. (Related follow-up lives in dependency-management.)
Case study: prism — commit lineage as primary source (as of 2026-07-04)
~/prism — the claim "security hardening exists and is tested" verifies directly against history: e264a72 → b7bbabb → 51b92c9 "security: add payload size limit, concurrency cap, repo name validation, Ollama timeout" → b011239 "tests: add security path coverage for signature, repo validation, size limit". Verified by git log --oneline in the repo, 2026-07-04. Contrast a properly labeled weaker claim from the same investigation set: ASVER's "Remove server build from package.json for static deployment" motive — strip what the deployment shape forbids — is partly inferred, and is recorded with that label rather than upgraded to fact.
Validation Criteria
You applied this skill correctly when:
- Every claim in your output carries one of the four epistemic labels (or an obvious verified citation).
- Each "verified" claim has a citation a reader can replay: path, hash, or command.
- At least one hypothesis was actively tested for disconfirmation — and the digest says what was ruled out.
- Volatile findings carry date stamps; each open item carries the exact command that would close it.
- The digest answers the step-1 question directly, and the dependent decision can be made from it without re-research.
- You stopped: the thread ends in an action or recommendation, not another search.
Provenance & Maintenance
- Sources:
~/ruflo,~/ragit,~/prism,~/asver— investigated 2026-07-04. Methodology is general; the repos supply worked examples of provenance discipline. The ASVER motive is explicitly (partly inferred), preserved as a live demonstration of honest labeling. - Assumptions: commit hashes (
a1ad63ed,0a06ae9b,e264a72,b7bbabb,51b92c9,b011239) and quoted commit messages are point-in-time facts from those repos' histories. The ruflo dist-tag commit is cited by message, not hash — locating its hash (requires verification):cd ~/ruflo && git log --oneline --all | grep -i "dist-tag". - Re-verification commands:
cd ~/ragit && git show --stat 0a06ae9b && grep -n "numpy\|chromadb" pyproject.toml setup.py 2>/dev/null cd ~/prism && git log --oneline -6 cd ~/ruflo && git log --oneline --all | grep -i "dist-tag\|alpha.3" - Likely to drift: npm registry/dist-tag state (per release), dependency pins (whenever chromadb/numpy move), commit tips, the ASVER inference (may be confirmed or refuted by its owner).
- Maintenance checklist:
- Re-run re-verification commands; re-stamp dates.
- Upgrade or delete the ASVER example once its motive is confirmed.
- Confirm cross-referenced skills (
codebase-onboarding,debugging-playbook,failure-archaeology,architecture-analysis,engineering-minimalism,dependency-management) still exist under those directory names.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

