agentleFS
junhjangjunhjang/claude-harness/.claude/skills/audit-docs-semantic/SKILL.md

audit-docs-semantic

Skill0 starsChanged 6 months agoMITView on GitHub

Bounded-fan-out semantic review of every doc under a subtree. Dispatches /review-doc against each .md file with concurrency ≤4, hard per-invocation cost cap, and a single combined audit output. Use this when reviewing a whole subtree — `/audit-docs-semantic docs/`, `/audit-docs-semantic .claude/`, `/audit-docs-semantic docs/harness/components/`.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 2 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All other paths → Sonnet default (~$0.08)

    line 35
  2. Very specific:

    Run /review-doc <file> for each enumerated file with bounded concurrency:

    line 49
  3. Very specific:

    Mid-flight cost tracking: after each dispatch returns, accumulate the actual cost (the RESULT: JSON includes the model used.

    line 52
  4. Very specific:

    Write .claude/audits/<YYYY-MM-DD>-semantic-audit-<subtree-slug>.md (suffixing -N on same-day collision).

    line 57
  5. Distinctive:

    Concurrency cap is hard: Do not raise above 4 even on operator request — the cap reflects API rate-limit shape, not just budget.

    line 93

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

/audit-docs-semantic

Fan-out wrapper around /review-doc. Reviews every .md file under <subtree> semantically, with a hard per-invocation cost cap and concurrency ≤4 to avoid daily-budget exhaustion.

This skill is operator-only (disable-model-invocation: true). It is the only safe way to fan out semantic review across a subtree — direct multi-doc dispatch via /review-doc is forbidden by the agent contract.

Scope vs the other auditors

/audit-docs (deterministic)/review-doc (single doc)/audit-docs-semantic (this skill)
Surfaceevery doc, structuralone doc, semanticevery doc under subtree, semantic
MethodD1–D11 (deterministic only)Sonnet/Opus full readFan-out of /review-doc per doc
Cost$0 (deterministic only)~$0.08 / ~$0.40 per doccapped at $5.00/invocation default
Output.claude/audits/<date>-docs-audit.mdinline session verdict.claude/audits/<date>-semantic-audit-<subtree-slug>.md

When invoked: /audit-docs-semantic <subtree>

1. Validate the subtree

Resolve <subtree> against the repo. If it does not exist or is not a directory, list closest matches and stop. Reject . (whole-repo) outright — the operator must scope intent.

2. Pre-flight cost estimate

Enumerate .md files under <subtree> (respecting the same exemptions as audit-docs: skip .claude/hooks/*.disabled, scratch/, etc.).

For each file, compute the dispatch model per /review-doc's heuristic:

  • Safety-critical paths (configured per project) → Opus override (~$0.40)
  • All other paths → Sonnet default (~$0.08)

Sum the estimate. Default per-invocation cap: max_invocation_cost_usd: 5.00. Consuming repos that want a configurable cap can wire it through their own config (e.g., config/cron/d4.yaml).

If estimate > cap:

  • Refuse. Print the file count, the estimate (with Sonnet/Opus split), and at least one suggested narrower subtree (e.g., "53 docs / $8.20 — try <subtree>/components/ (24 docs / $4.10) instead").
  • The operator can override by passing --cap <usd>; never auto-raise.

3. Operator confirmation

Show the count + estimate + concurrency setting. Default: ask before launching. Skip the confirm only when the operator explicitly passes --yes.

4. Fan-out dispatch

Run /review-doc <file> for each enumerated file with bounded concurrency:

  • Concurrency cap: ≤4. Higher concurrency risks rate-limit thrashing on the Anthropic API and breaks the cost-tracking arithmetic.
  • Mid-flight cost tracking: after each dispatch returns, accumulate the actual cost (the RESULT: JSON includes the model used; map to per-invocation $). If cumulative > cap, abort the remaining dispatches; emit a partial audit with the dispatched-so-far results plus an EARLY-TERMINATED marker.
  • Failure handling: a single dispatch failing (prompt missing, doc unreadable) does not abort the run. Capture the error in the per-doc record; continue.

5. Combine into a single audit doc

Write .claude/audits/<YYYY-MM-DD>-semantic-audit-<subtree-slug>.md (suffixing -N on same-day collision). Frontmatter type: AUDIT. Sections:

# Semantic audit — <subtree> — <date>

Method: /audit-docs-semantic
Files dispatched: <n>
Concurrency: <≤4>
Cumulative cost: $<x> / cap $<cap>
Status: completed | EARLY-TERMINATED

## Per-doc verdicts

| Doc | Type | Model | Verdict | B / CR / N / A |
|---|---|---|---|---|

## Aggregated blockers (<n>)
... (each finding cites <doc-path>:<line>)

## Aggregated change-requests (<n>)
...

## Aggregated ambiguous (<n>)
...

## Errors / skipped (<n>)
...

6. Surface to operator

Report counts: dispatched / blockers / change-requests / nits / ambiguous / errors. Do not auto-create issues. Do not auto-edit any doc.

Rules

  • Per-invocation cost cap is mandatory. No --cap-disable flag. The cap is configurable but always enforced.
  • Concurrency cap is hard. Do not raise above 4 even on operator request — the cap reflects API rate-limit shape, not just budget.
  • No auto-issue creation. Findings are read-only; the operator decides what to escalate.
  • Honour the dispatch model heuristic from /review-doc — do not bulk-promote everything to Opus.
  • Output is ONE audit doc per invocation, not one per dispatched file. The point of fan-out is consolidated review.

Forbidden

  • Dispatching against the whole repo (.) — the operator must scope.
  • Concurrency >4.
  • Auto-raising the cost cap when the estimate exceeds it.
  • Skipping the operator confirmation step (except via explicit --yes).
  • Modifying any doc under the dispatched subtree.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.