audit-docs-semantic
Skill0 starsChanged 6 months agoMITView on GitHub
Bounded-fan-out semantic review of every doc under a subtree. Dispatches /review-doc against each .md file with concurrency ≤4, hard per-invocation cost cap, and a single combined audit output. Use this when reviewing a whole subtree — `/audit-docs-semantic docs/`, `/audit-docs-semantic .claude/`, `/audit-docs-semantic docs/harness/components/`.
In this repository18 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- audit-docsSkill
- audit-harness-coverageSkill
- budget-statusSkill
- cavemanSkill
- configurationSkill
- critic-reviewSkill
- doc-patternSkill
- error-taxonomySkill
- grill-meSkill
- grill-with-docsSkill
- new-docSkill
- python-failure-modesSkill
- review-changesSkill
- review-docSkill
- rust-failure-modesSkill
- verification-before-completionSkill
- zoom-outSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 2 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 35
All other paths → Sonnet default (~$0.08)
- Very specific: line 49
Run /review-doc <file> for each enumerated file with bounded concurrency:
- Very specific: line 52
Mid-flight cost tracking: after each dispatch returns, accumulate the actual cost (the RESULT: JSON includes the model used.
- Very specific: line 57
Write .claude/audits/<YYYY-MM-DD>-semantic-audit-<subtree-slug>.md (suffixing -N on same-day collision).
- Distinctive: line 93
Concurrency cap is hard: Do not raise above 4 even on operator request — the cap reflects API rate-limit shape, not just budget.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
/audit-docs-semantic
Fan-out wrapper around /review-doc. Reviews every .md file under <subtree> semantically, with a hard per-invocation cost cap and concurrency ≤4 to avoid daily-budget exhaustion.
This skill is operator-only (disable-model-invocation: true). It is the only safe way to fan out semantic review across a subtree — direct multi-doc dispatch via /review-doc is forbidden by the agent contract.
Scope vs the other auditors
/audit-docs (deterministic) | /review-doc (single doc) | /audit-docs-semantic (this skill) | |
| Surface | every doc, structural | one doc, semantic | every doc under subtree, semantic |
| Method | D1–D11 (deterministic only) | Sonnet/Opus full read | Fan-out of /review-doc per doc |
| Cost | $0 (deterministic only) | ~$0.08 / ~$0.40 per doc | capped at $5.00/invocation default |
| Output | .claude/audits/<date>-docs-audit.md | inline session verdict | .claude/audits/<date>-semantic-audit-<subtree-slug>.md |
When invoked: /audit-docs-semantic <subtree>
1. Validate the subtree
Resolve <subtree> against the repo. If it does not exist or is not a directory, list closest matches and stop. Reject . (whole-repo) outright — the operator must scope intent.
2. Pre-flight cost estimate
Enumerate .md files under <subtree> (respecting the same exemptions as audit-docs: skip .claude/hooks/*.disabled, scratch/, etc.).
For each file, compute the dispatch model per /review-doc's heuristic:
- Safety-critical paths (configured per project) → Opus override (~$0.40)
- All other paths → Sonnet default (~$0.08)
Sum the estimate. Default per-invocation cap: max_invocation_cost_usd: 5.00. Consuming repos that want a configurable cap can wire it through their own config (e.g., config/cron/d4.yaml).
If estimate > cap:
- Refuse. Print the file count, the estimate (with Sonnet/Opus split), and at least one suggested narrower subtree (e.g., "53 docs / $8.20 — try
<subtree>/components/(24 docs / $4.10) instead"). - The operator can override by passing
--cap <usd>; never auto-raise.
3. Operator confirmation
Show the count + estimate + concurrency setting. Default: ask before launching. Skip the confirm only when the operator explicitly passes --yes.
4. Fan-out dispatch
Run /review-doc <file> for each enumerated file with bounded concurrency:
- Concurrency cap: ≤4. Higher concurrency risks rate-limit thrashing on the Anthropic API and breaks the cost-tracking arithmetic.
- Mid-flight cost tracking: after each dispatch returns, accumulate the actual cost (the
RESULT:JSON includes the model used; map to per-invocation $). Ifcumulative > cap, abort the remaining dispatches; emit a partial audit with the dispatched-so-far results plus anEARLY-TERMINATEDmarker. - Failure handling: a single dispatch failing (prompt missing, doc unreadable) does not abort the run. Capture the error in the per-doc record; continue.
5. Combine into a single audit doc
Write .claude/audits/<YYYY-MM-DD>-semantic-audit-<subtree-slug>.md (suffixing -N on same-day collision). Frontmatter type: AUDIT. Sections:
# Semantic audit — <subtree> — <date>
Method: /audit-docs-semantic
Files dispatched: <n>
Concurrency: <≤4>
Cumulative cost: $<x> / cap $<cap>
Status: completed | EARLY-TERMINATED
## Per-doc verdicts
| Doc | Type | Model | Verdict | B / CR / N / A |
|---|---|---|---|---|
## Aggregated blockers (<n>)
... (each finding cites <doc-path>:<line>)
## Aggregated change-requests (<n>)
...
## Aggregated ambiguous (<n>)
...
## Errors / skipped (<n>)
...6. Surface to operator
Report counts: dispatched / blockers / change-requests / nits / ambiguous / errors. Do not auto-create issues. Do not auto-edit any doc.
Rules
- Per-invocation cost cap is mandatory. No
--cap-disableflag. The cap is configurable but always enforced. - Concurrency cap is hard. Do not raise above 4 even on operator request — the cap reflects API rate-limit shape, not just budget.
- No auto-issue creation. Findings are read-only; the operator decides what to escalate.
- Honour the dispatch model heuristic from
/review-doc— do not bulk-promote everything to Opus. - Output is ONE audit doc per invocation, not one per dispatched file. The point of fan-out is consolidated review.
Forbidden
- Dispatching against the whole repo (
.) — the operator must scope. - Concurrency >4.
- Auto-raising the cost cap when the estimate exceeds it.
- Skipping the operator confirmation step (except via explicit
--yes). - Modifying any doc under the dispatched subtree.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

