verify-docs-findable
Skill15 starsChanged 5 months agoMITView on GitHub
Audit whether a repo's docs actually ANSWER the questions a reader has — by spawning fresh, cheap (Haiku) agents that cold-read ONLY the docs and measuring how fast they reach the answer, whether they hit dead-ends, whether they fall back to source code, and whether they cite docs that contradict each other. Use after a doc reorg, when docs "feel scattered," or when the same confusion keeps recurring. Surfaces findability gaps (a corpus can be COMPLETE — every doc indexed — yet not FINDABLE) plus a prioritized fix list. Works on any repo's docs, not just this one.
In this repository24 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- examples/CLAUDE.mdCLAUDE.md
- src/CLAUDE.mdCLAUDE.md
- add-a-linterSkill
- audience-checkSkill
- audit-feedback-loopSkill
- code-qualitySkill
- deep-researchSkill
- dogfood-cliSkill
- enforce-rules-formatSkill
- generate-logoSkill
- hexagonal-architectureSkill
- landing-siteSkill
- pr-to-lint-ruleSkill
- review-docsSkill
- screenshotSkill
- ship-a-featureSkill
- pr-to-lint-ruleSkill
- adopt-specSkill
- debug-my-harnessSkill
- edit-specSkill
- linter-docsSkill
- strengthenSkill
- test-harnessSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 29
Test the recurring, load-bearing, and mis-stated questions — the ones that actually cost time.
- Distinctive: line 35
If none given: ask the user for the questions, or infer them from the corpus's top-level topics.
- Very specific: line 42
Use the Agent tool with model: "haiku", run_in_background: true, one per question.
- Distinctive: line 54
Run all in one message so they go concurrently.
- Very specific: line 95
Keep the question set in the repo (e.g.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
verify-docs-findable — cold-read the docs with fresh eyes
The insight this operationalizes: complete ≠ findable. An index that lists every doc (and even passes a "every doc is indexed" check) can still make a reader take 5 hops and fall back to source code to answer a basic question. The only honest test is to give the question to someone who has never seen the repo and watch how far they get from the docs alone. A fresh cheap agent is that someone.
Why HAIKU, and why "docs only"
- Haiku (a small, cheap model), fresh (no session context). The point is a naive cold reader, NOT a clever one. A strong model with your context will grep the source, reason around gaps, and hide the doc problem. Haiku, told to use only the docs, exposes it.
- "Use ONLY the docs, start at the index." The moment a verifier has to open source code to answer, the docs have failed — that's the signal. Steps-from-index is the metric.
Procedure
1. Get the questions (3–8)
Test the recurring, load-bearing, and mis-stated questions — the ones that actually cost time. Sources, in order:
- What the user (or you) just got confused about this session — the highest-signal source.
- The "canonical answers" the docs claim to provide (a docs corpus usually has ~5–8 facts it exists to convey; test those).
- If none given: ask the user for the questions, or infer them from the corpus's top-level topics. Don't invent trivia — test what a real contributor needs.
Phrase each as a real question a reader would ask, not a doc title.
2. Spawn one fresh Haiku verifier per question (in parallel)
Use the Agent tool with model: "haiku", run_in_background: true, one per question. Give
each the SAME strict template (fill in <QUESTION>):
You are a fresh engineer opening
<repo path>for the first time. Answer using ONLY the repo's docs — START at the index (<the index file(s)>) and follow pointers; prefer docs over reading source code. QUESTION:<QUESTION>REPORT: (a) your answer; (b) the exact file(s) where you found it; (c) how many steps from the index (did a pointer take you straight there, or did you dig?); (d) clear/unambiguous or scattered/confusing? (e) did you have to read source code because the docs didn't say? Be honest — this tests whether the docs make this findable.
Run all in one message so they go concurrently. (Haiku + parallel = the audit is cheap.)
3. Score each answer
| Signal | Good | Gap |
| Correct? | matches ground truth | wrong/partial → the doc is wrong or missing |
| Steps from index | 1–2 | 3+ → no signposted pointer |
| Dead-ends | none | landed on a wrong-but-plausible doc first → title/scoping is misleading |
| Needed source code? | no | yes → the docs don't actually state it |
| Contradictions | — | cited two docs that disagree → a cohesion bug (fix immediately) |
Verifiers surface contradictions for free — a cold reader citing two docs that say different things is the cheapest contradiction-finder you have.
4. Fix the gaps
- 3+ steps / dead-end → add a question→doc pointer at the index entry point (a "Canonical answers: question → the ONE doc" block). Naming the answer is what turns 5 hops into 1.
- Needed source code → the fact isn't in prose; write it into the one canonical doc.
- Wrong answer → the doc is stale/incorrect; fix it (append-don't-erase if it records a decision).
- Contradiction → reconcile the docs; keep the historical decision, mark it superseded.
- Scattered across N docs → consolidate into ONE doc per question; make the others point to it.
5. Re-verify (measure, don't assume)
Re-run the worst question(s) against the fixed docs. Confirm the number moved (e.g. 5 steps → 2). A fix you didn't re-measure is a guess.
Output
A short findability scorecard (question | correct | steps | needed-source | verdict) + a prioritized fix list. Report the before/after on any question you fixed and re-verified.
Notes
- This is orthogonal to "is every doc indexed?" completeness checks — it measures the layer above: can a stranger reach the answer. Run it after any doc reorganization.
- Keep the question set in the repo (e.g. a
docs-findability-questions.md) so the audit is repeatable and the canonical answers stay honest as the corpus grows. - Scales down (1 question, 1 agent, to spot-check one fix) and up (the full canonical set).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

