Auditing plugin self-consistency

This plugin teaches Claude to use its own MCP tools through a scattered set of hand-maintained files — skills, agent definitions, rules, hooks, docs/*.md, CLAUDE.md, README.md. None is generated, so any of them can silently drift the moment something changes and the doc edit is skipped. This is the audit pass that catches that, run as its own task rather than trusted to unrelated work.

Three sources of truth, and which one wins

The code is not the only ground truth, and treating it that way is how a defect gets locked in: the doc gets rewritten to match code that was itself wrong.

• Code is ground truth for facts. Tool names, signatures, arguments, defaults, return shapes, error codes, what a hook actually dispatches. A doc that disagrees here always loses.
• Official Claude documentation is ground truth for design. How a tool gets found, what is always loaded and what is not, what a skill / agent / rule may contain, the harness's own limits. Code that disagrees here loses — and the result is a CODE FINDING, not a doc edit.
• Internal documents are ground truth for nothing. They are claims, checked against both.

The three reference files

All in ${CLAUDE_PLUGIN_ROOT}/skills/dotnet-consistency/, read on demand — a run that only needs drift detection should not pay for the rest.

Step 0 — enumerate the actual tool surface

Get the current, complete list from the source — don't trust any doc's tool list, including this skill's own examples; that is exactly the kind of claim being audited. Not grep: the guard-cs-bash-read hook flags a shell read of a solution .cs file as advisory (allowed, not blocked, since 2026-08-21) rather than denying it, but get_symbol is still the right call regardless of what the hook does — this is a symbol query, not a text search.

get_symbol(symbols: ["ContextTools", "FlowTools", "GraphTools", "HistoryTools",
                     "RenameTools", "MetricsTools", "ServerTools", "GuardTools"], include: "members")

Each [McpServerToolType] class's public methods are its tools, one per method, with every parameter and default. Confirm the class list itself first with search_index(query: "Tools", pathPrefix: "src/DotnetToolkit.McpServer/Tools"), so a new tool group is caught rather than assumed away — a new Tools/*.cs file is itself a finding (Step 7). That search is also what keeps the list above honest: it is an example, and an example of the thing being audited goes stale exactly like any other doc claim.

Not every file under Tools/ is a tool group. ToolTelemetry.cs, ResponseGuard.cs and VocabularyHint.cs carry no [McpServerToolType] and are shared internal helpers; they belong in docs/design/architecture.md's Tools/ table, not in any tool list. Check the attribute, not the folder. A search hit's modifiers column separates them at a glance without a second call: a tool group renders ps and an A1 in shape (the [McpServerToolType] attribute), a helper renders is with no A.

The audit, step by step

1. Harness compliance. Read harness-compliance.md and run §A–§G. First, because it is the only axis that can produce code findings, and because a guidance change invalidates a previously-clean result on every other axis.

Before running it, apply the staleness gate: if claude-docs.md's last-verified is older than its refresh-window, or the user asked to refresh the links, run that file's refresh procedure and update the shortlists before checking anything against them.

2. Implementation. For each tool from Step 0, read its method body (not just the [Description]) — get_symbol for its declarationSites, then a ranged Read of that span — and confirm what it actually accepts, returns, and errors on. Docs get audited against this, not against their own prior text.

3. [Description] attributes vs. documented behavior and return shape. Compare each tool's [Description] (on the method and on each parameter) against:

• docs/tools/<tool>.md — its "When to reach for it" guidance, arguments, defaults, the example call/response, and its Next steps footer. If you can, call the tool with the documented example arguments and confirm the response still matches what's printed; an example that no longer matches the current return shape is worse than no example.
• Any wording that has drifted from what the code now does — e.g. a default named in the attribute that the method body no longer honors.
• Whether it works as a search target — that criterion, its thresholds and the live probe are harness-compliance.md §B and §C, already run in Step 1. Don't re-derive them here.

4. Skills vs. tool set. For skills/dotnet-read/SKILL.md, skills/dotnet-write/SKILL.md, skills/dotnet-explore/SKILL.md, skills/dotnet-review/SKILL.md and skills/dotnet-architecture/SKILL.md, confirm every tool they describe still exists (Step 0) with the arguments they claim, and that every tool relevant to that skill's subject actually appears in it — a tool added to Tools/*.cs that fits an existing skill's scope but isn't mentioned there is a finding, not just a missing doc row.

Every one of the 17 tools must be named by some skill, because .claude/rules/dotnet-index.md names none. A tool named by no skill is unreachable: nothing always-loaded points at it, and free-text ToolSearch does not reliably find it (§C). Split by side — retrieval and the server/meta tools in dotnet-read, rename_symbol in dotnet-write — with workspace_status and reload_workspace legitimately in both, since each skill's step 0 depends on them, and set_hook_guards in dotnet-performance alone, its only sanctioned caller.

set_hook_guards is the one deliberate exception, owned by dotnet-performance alone. It is not missing from dotnet-read; it is kept out. The tool suspends the guards that make the rest of the plugin load-bearing, so naming it in the skill every read task loads would advertise the off-switch to every session — including the ones the guards exist for. Reachability is satisfied because dotnet-performance names it (for its strict variant only — the default procedure calls it nowhere, because Claude Code's auto-mode classifier denies the call as self-modification and the meter's wholeFileCsReads count answers the same question without it), and that skill is the only sanctioned reason to call it. If a future audit finds it added to dotnet-read or to the always-loaded rule, or back in dotnet-performance's default steps, that is the finding — not its absence there.

5. Every instruction/guideline file that tells a caller to use the MCP tools. Check each still lists every tool from Step 0, with nothing stale (a tool it describes that no longer exists) and nothing missing (a tool that exists but appears nowhere in it):

5b. Consumer reachability — does the plugin work out of the box? Steps 2–5 check that the files describing the tools are accurate. This checks they are reachable from a consuming repo, where this repo's CLAUDE.md does not exist. Every operational instruction must live in something that ships: the one rule init copies (.claude/rules/dotnet-index.md), a skill, an agent file, or a docs//standards/ file reachable by ${CLAUDE_PLUGIN_ROOT} path from a skill — rules and agent files must not depend on that variable expanding, which is why agents/dotnet-code-review.md resolves its own pluginRoot from its own workspace_status call instead of expecting a path handed to it.

The install procedure is half of this step. dotnet-init is prose, hand-maintained, and describes a file set that grows every time this plugin gains a tool, doc, skill, or standard. When it falls behind, the failure is silent in both directions: an asset ships but never reaches the consumer, or the uninstall instructions leave files behind that keep steering a repo that no longer has the plugin. Read docs/install/audit.md and run it. Two of its findings are hard errors worth naming here because both have shipped: a ${CLAUDE_PLUGIN_ROOT} path in a copied rule, and any claim that mechanism-1 assets need a repo-local install or cleanup step.

Then two sweeps, both cheap:

• This repo's CLAUDE.md, paragraph by paragraph. For each instruction, decide: is this about maintaining the plugin (correct — it belongs only here), or about using the tools? Anything in the second category must also exist in a shipped file, with CLAUDE.md carrying a pointer rather than a second copy. The write-path discipline is the canonical example — the "Edit/Write are the write path, the hook re-indexes, dotnet build is the gate" argument lives in the rule and dotnet-write, never in CLAUDE.md, because a consumer never sees CLAUDE.md.
• The maintainer's memory directory (~/.claude/projects/<repo-slug>/memory/, indexed by its MEMORY.md). Read the index, then each entry, and sort every one into: (a) operational for consumers → must be embedded in a shipped file; (b) operational for this repo only → belongs in CLAUDE.md or .claude/rules/; (c) personal or environment-specific → belongs in neither, and is correctly only in memory. Memory is user-local and does not travel with the plugin, so anything in (a) or (b) that exists only there is a finding — silently absent for every other user, and for this repo after a memory reset.

Report each finding as: the memory or CLAUDE.md paragraph, its category, and the shipped file that should carry it. Do not copy a category-(c) fact into a shipped file to close a finding — categorising it correctly is closing it.

6. Hooks and launch path. All seven hooks are hook <name> subcommands of the published server binary, in src/DotnetToolkit.McpServer/Hooks/ — HookCli.cs dispatches, the GuardCsRead/GuardCsBashRead/CsFileRefresh/CsWriteChecklistHint/ToolCallMeter files carry the behaviour, and four more have no hooks.json entry of their own: CsFileMembership.cs/ BashCommandScanner.cs (shared guard logic, including the ranged-read recogniser), GuardSuspension.cs (the off-switch HookCli reads before dispatching the two read guards, written by set_hook_guards) and ControlClient.cs (how refresh-cs-files and meter-tool-call both reach the running server, with the command words in Control/ControlCommands.cs). Read those with get_symbol (not grep — a tree grep here would only earn an advisory note from guard-cs-bash-read, not a denial, but get_symbol is still the right call), plus hooks/hooks.json and .mcp.json:

• Does each guard's advisory text still name the correct tool(s) and procedure (get_symbol plus a ranged Read for the read guards, reload_workspace(scope: "all") for the refresh hook's unreachable-server note)? It is read at the exact moment a caller is nudged — a stale one teaches the wrong fix at the worst moment. And do both read guards still pass a ranged read silently in every mode (Read with offset/limit; sed -n 'a,bp', head -n, tail -n +N)? A guard that notes a ranged read is arguing against the route the skills recommend.
• hint-cs-write-checklist owns the write-time checklist, matching Edit/Write/ NotebookEdit. Its text must still match the standards it summarizes (security.md, testing.md), and it must stay once-per-session and fail-quiet — a version that fires on every edit, or that denies, is a finding.
• refresh-cs-files is the PostToolUse hook that makes the index follow edits. It must stay silent on success, send every written .cs path (from Edit/Write/NotebookEdit/apply_patch input and from Bash write shapes) to the server's refresh command, ask for a rescan on a git verb, and answer with a note only when a file was added/deleted (reload started), a refresh is queued, or the server was unreachable.
• meter-tool-call is the one hook matching every tool ("*"), and the one exempt from GuardSuspension on purpose. It observes rather than withholds, and a guard suspension is asked for precisely in order to measure the unguarded route — a version that honours suspension silently produces a benchmark with half its data missing, which is the finding. It is also the hook that records wholeFileCsRead on every call, the count dotnet-performance reads per probe instead of switching guard state; the predicate is WholeFileCsRead.Matches, shared with the two read guards so the count and the advisory agree on what a whole-file read is — a version where they diverge is the finding. It must also stay fail-open-and-silent and keep reporting over the control channel rather than opening the store itself: a hook process has its own ambient session id, so a row it wrote directly would be invisible to every read.
• Does hooks/hooks.json name subcommands HookCli actually dispatches, with matchers (Edit/Write/NotebookEdit/apply_patch/Read/Bash) matching docs/design/hooks.md and docs/design/architecture.md's "Packaging" section? The shell-free and output-cap invariants are harness-compliance.md §G. scripts/ holds developer conveniences only; anything new there or in Hooks/ unmentioned by docs/design/hooks.md or "Packaging" is a Step 7 finding.
• Matchers key on tool name, not on which agent calls, so they fire for subagents too. For dotnet-code-review: (a) its tools: list never grants Edit/Write/NotebookEdit — memory: project makes the harness grant them anyway, which is why Boundaries carries the weight; (b) its Process step 2 still locates with search_index/get_symbol and reads with a ranged Read over declarationSites, treating a whole-file Read as the scope-mode exception rather than the default — its own text (checked in Step 5's table) must say that a ranged read draws no note and a whole-file one only advises. For dotnet-explore, whose own file bans Read on .cs outright with no hook backstop behind that ban, a Read of a .cs file appearing in its transcript is itself the drift to report — the file's own instruction, not any hook, is what should have stopped it.

7. Internal drift. Read drift.md and run it. This is the step that starts from what changed rather than from a file list: get_semantic_diff against a baseline, then git log for the recorded intent and for everything the semantic diff cannot see, checked against the drift-sensitivity ranking. It is how a shipped-but-undocumented tool gets found — nothing in a per-file check ever will, because the files that should mention it are internally consistent.

8. The skills' own instructions. Once Steps 1–7 have surfaced concrete drift, the fix usually touches a skill body, not just a table row — e.g. a new tool needs a section in dotnet-read or dotnet-write (whichever side it belongs to), a row in that skill's cheap-route table if it displaces an existing route, and its own docs/tools/<tool>.md. It needs nothing in .claude/rules/dotnet-index.md, which names no tools: a new row there is only warranted by a new skill. Update the skill body, not only its tool list, so a caller reading the skill gets the same guidance a caller reading the code would.

9. CLAUDE.md and README.md last. The two files a fresh session or a new user reads first, so they should reflect the already-corrected state of everything above rather than being patched independently of it. Update CLAUDE.md per Step 5's table, then README.md's Features table and prose.

Output format

Report drift the same way dotnet-code-review reports findings — concrete and file-anchored, not a narrative:

• File:line of the stale claim.
• What it claims vs. what is actually true, naming which source of truth decided it — the code (Steps 0/2) or the official guidance (claude-docs.md, via Step 1).
• Exact fix — the replacement text or the specific file to add a row to. Point at the file; don't describe the fix in the abstract.

A finding driven by guidance rather than by code, where the code is what must change, uses a distinct form so it is never mistaken for something this skill already fixed:

CODE FINDING — src/…/Tools/X.cs:NN · guideline: harness-compliance.md §B · current: <measurement or text> · required: <threshold> · proposed replacement: <exact text> · apply via: dotnet-write → Edit, then dotnet build.

Group findings by file, in Step 5's table order, then hooks, then CLAUDE.md/README.md last, with CODE FINDINGs collected together at the end. State the baseline Step 7 used. If a section is in sync, say so in one line — don't manufacture findings to justify the run.

Fixing vs. reporting

• Doc findings: fix them yourself, in Step 1→9 order. Every such fix is to a non-.cs file, so no hook fires and no build is needed.
• CODE FINDINGs: report only, never apply. This skill does not edit .cs, even when the fix is a one-line attribute change with no behavior impact — an audit that can change the tool contract can no longer be run freely. Hand it off with the exact replacement text; the user or a follow-up dotnet-write task applies it.
• Never silently skip a step because "nothing looks wrong there" — state it was checked and came back clean. A skipped step and a clean step read identically in a report, and only one of them is true.