agentleFS
zerniezernie/vigiles/.claude/skills/verify-docs-findable/SKILL.md

verify-docs-findable

Skill15 starsChanged 5 months agoMITView on GitHub

Audit whether a repo's docs actually ANSWER the questions a reader has — by spawning fresh, cheap (Haiku) agents that cold-read ONLY the docs and measuring how fast they reach the answer, whether they hit dead-ends, whether they fall back to source code, and whether they cite docs that contradict each other. Use after a doc reorg, when docs "feel scattered," or when the same confusion keeps recurring. Surfaces findability gaps (a corpus can be COMPLETE — every doc indexed — yet not FINDABLE) plus a prioritized fix list. Works on any repo's docs, not just this one.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Test the recurring, load-bearing, and mis-stated questions — the ones that actually cost time.

    line 29
  2. Distinctive:

    If none given: ask the user for the questions, or infer them from the corpus's top-level topics.

    line 35
  3. Very specific:

    Use the Agent tool with model: "haiku", run_in_background: true, one per question.

    line 42
  4. Distinctive:

    Run all in one message so they go concurrently.

    line 54
  5. Very specific:

    Keep the question set in the repo (e.g.

    line 95

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

verify-docs-findable — cold-read the docs with fresh eyes

The insight this operationalizes: complete ≠ findable. An index that lists every doc (and even passes a "every doc is indexed" check) can still make a reader take 5 hops and fall back to source code to answer a basic question. The only honest test is to give the question to someone who has never seen the repo and watch how far they get from the docs alone. A fresh cheap agent is that someone.

Why HAIKU, and why "docs only"

  • Haiku (a small, cheap model), fresh (no session context). The point is a naive cold reader, NOT a clever one. A strong model with your context will grep the source, reason around gaps, and hide the doc problem. Haiku, told to use only the docs, exposes it.
  • "Use ONLY the docs, start at the index." The moment a verifier has to open source code to answer, the docs have failed — that's the signal. Steps-from-index is the metric.

Procedure

1. Get the questions (3–8)

Test the recurring, load-bearing, and mis-stated questions — the ones that actually cost time. Sources, in order:

  • What the user (or you) just got confused about this session — the highest-signal source.
  • The "canonical answers" the docs claim to provide (a docs corpus usually has ~5–8 facts it exists to convey; test those).
  • If none given: ask the user for the questions, or infer them from the corpus's top-level topics. Don't invent trivia — test what a real contributor needs.

Phrase each as a real question a reader would ask, not a doc title.

2. Spawn one fresh Haiku verifier per question (in parallel)

Use the Agent tool with model: "haiku", run_in_background: true, one per question. Give each the SAME strict template (fill in <QUESTION>):

You are a fresh engineer opening <repo path> for the first time. Answer using ONLY the repo's docs — START at the index (<the index file(s)>) and follow pointers; prefer docs over reading source code. QUESTION: <QUESTION> REPORT: (a) your answer; (b) the exact file(s) where you found it; (c) how many steps from the index (did a pointer take you straight there, or did you dig?); (d) clear/unambiguous or scattered/confusing? (e) did you have to read source code because the docs didn't say? Be honest — this tests whether the docs make this findable.

Run all in one message so they go concurrently. (Haiku + parallel = the audit is cheap.)

3. Score each answer

SignalGoodGap
Correct?matches ground truthwrong/partial → the doc is wrong or missing
Steps from index1–23+ → no signposted pointer
Dead-endsnonelanded on a wrong-but-plausible doc first → title/scoping is misleading
Needed source code?noyes → the docs don't actually state it
Contradictions—cited two docs that disagree → a cohesion bug (fix immediately)

Verifiers surface contradictions for free — a cold reader citing two docs that say different things is the cheapest contradiction-finder you have.

4. Fix the gaps

  • 3+ steps / dead-end → add a question→doc pointer at the index entry point (a "Canonical answers: question → the ONE doc" block). Naming the answer is what turns 5 hops into 1.
  • Needed source code → the fact isn't in prose; write it into the one canonical doc.
  • Wrong answer → the doc is stale/incorrect; fix it (append-don't-erase if it records a decision).
  • Contradiction → reconcile the docs; keep the historical decision, mark it superseded.
  • Scattered across N docs → consolidate into ONE doc per question; make the others point to it.

5. Re-verify (measure, don't assume)

Re-run the worst question(s) against the fixed docs. Confirm the number moved (e.g. 5 steps → 2). A fix you didn't re-measure is a guess.

Output

A short findability scorecard (question | correct | steps | needed-source | verdict) + a prioritized fix list. Report the before/after on any question you fixed and re-verified.

Notes

  • This is orthogonal to "is every doc indexed?" completeness checks — it measures the layer above: can a stranger reach the answer. Run it after any doc reorganization.
  • Keep the question set in the repo (e.g. a docs-findability-questions.md) so the audit is repeatable and the canonical answers stay honest as the corpus grows.
  • Scales down (1 question, 1 agent, to spot-check one fix) and up (the full canonical set).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.