Auditing Site Navigation

Overview

Agents already audit sites well. Left alone they read routing config correctly, verify claims at file:line, reproduce failure states live, and refuse to invent facts about the owner. This skill teaches none of that.

What they do not build unprompted is a measurement instrument. They invent personas mid-run, score by gestalt, label an unweighted mean "weighted", and persist nothing. The findings are reproducible; the numbers are not. Any later "we went from 55 to 78" is then unfalsifiable, and a regression introduced while fixing goes undetected.

Core principle: the observations are the easy half. A score is only worth stating if a different agent, next month, can recompute it.

Step 1: copy the workflows in. Do this before reading further.

mkdir -p <target-repo>/.claude/workflows
cp <this-skill-dir>/workflows/*.js <target-repo>/.claude/workflows/
ls <target-repo>/.claude/workflows/          # confirm three files

Do this first, literally. In testing, an agent read this skill, agreed with it, hand-followed the Finding Contract from memory — and never copied the workflows. Its output then failed on exactly the fields the schema enforces: a cluster table with three undetected file-ownership collisions, three defects deducting zero points, and a baseline whose shape did not match what the rescore consumes. It complied with everything it could remember and missed everything it could not. Copy the files.

Every field is overridable via args; defaults describe an Astro-on-Vercel site.

Why this is not optional. The workflows enforce as required JSON schema fields the exact things that get skipped when an agent works from prose: the per-finding probe, the root-cause layer, the per-persona deduction ledger, and the coverage denominator. A required field cannot be quietly omitted. In testing, an agent given only prose guidance produced a scorecard whose deduction points summed to a negative score, and designed a baseline file it then never wrote to disk. The schema is the enforcement; the prose below is the rationale.

If the workflows genuinely cannot run, follow the Finding Contract below by hand — and expect to check your own arithmetic, because nothing else will.

The Finding Contract

Every finding carries all of these. A finding missing any one is not a finding.

id              F-014                    stable; the deduction ledger cites it
kind            defect | open-decision   open-decision scores ZERO points
severity        blocks-transaction | degrades-persona | cosmetic
confidence      verified-live | read-from-source | inferred
rootCauseLayer  markup|style|data-query|schema|routing|content
probe           ONE executable line
currentOutput   what that line prints right now, verbatim
files           exact paths, for the work partition

rootCauseLayer before proposedFix, always. If the symptom is a rendered value, trace it: component → props → server query → schema. State where the value actually comes from. A hardcoded-looking "7 of 12" was, in testing, a Convex query result with a static fallback — the prescribed copy fix would have contradicted the live database and permanently hidden the real fault. Any fix that makes a live fault unobservable is itself a finding.

Probes bind to the mechanism, not to text near it. grep -n 'dev=1' X.tsx matched only a code comment; the real gate was get("dev") === "1" elsewhere in the file. Deleting the comment would pass the probe while the hole stayed open. Before recording a probe, ask: if someone fixed only what this line matches, would the defect be gone? Prefer, in order: a grep with -c over the mechanism, an HTTP status, a built-output assertion, a browser expression.

Run every probe and paste its real output. A probe whose currentOutput was predicted rather than executed is worthless — in testing, 3 of 23 predicted outputs were wrong, one because the source used an en-dash.

Scoring

Print the weight vector before the table, justified against who the site is for, and show the multiplication. Never label an average "weighted" without printing weights. An unweighted mean lets the persona nobody sells to offset the one who signs the cheque.

Deduction ledgers are per-persona, not global. Each persona starts every dimension at 100 and deducts against finding IDs, scaled by severity. Points per dimension must equal 100 minus that dimension's score, for that persona. One global points vector cannot reconstruct four different persona score rows — if you have N personas you need N ledgers. Sum them yourself before publishing; this gets checked.

Counts over stated denominators. Never an invented percentage. dead footer links 2/2 → 0/2 is auditable; 34 → 61 is not.

Say plainly what the score is and is not. This instrument buys reconstructibility, not objectivity. The deductions sum correctly and every one names a finding, but their magnitudes are judgement calls. Put that sentence in the report. Without it, "68 → 81" gets quoted next quarter as an instrument reading, and the countable half — orphans, confirmed 404s, single-entry routes, probe pass counts — is the only part that ever deserved that status.

Order the report by severity, and put severity in the tables. A reader of the prose alone must be able to see that a one-way trapdoor outranks a stale placeholder. Every defect deducts non-zero points; a defect worth nothing is either an open-decision or a deduction you forgot.

Coverage

Enumerate before judging, and count every route type. Page files are not the route list: API handlers, dynamic segments, catch-alls, middleware-matched paths and static assets served from a public directory are all reachable surfaces. In testing, an audit reported "12 of 12 local routes" against an actual 9 page files plus 5 route handlers — a fabricated denominator that made an incomplete sweep look exhaustive, with five handlers never mentioned.

Build the checklist first, mark each item examined, and report the real ratio. When a defect class appears once, grep the whole tree for siblings and make that grep the probe.

The Re-Audit Firewall

Name and forbid the audit report, the fix list, the changelog and git history. An agent that knows what it was trying to achieve scores the intent, not the artifact.

The rescore replays recorded probes before anyone forms an impression, and reports fixed / still-broken / moved / unrunnable. moved is the one that matters: a rule deleted from one selector and reintroduced in another passes inspection and fails its probe.

Freeze personas as reusable assets — the workflows carry a default set. A new cast each run makes cross-run numbers incomparable even in principle.

Partitioning Fixes for Parallel Agents

Agents run concurrently against one working tree. Two writing the same file do not merge — the second silently destroys the first's work, and nothing surfaces it.

Worktree isolation is the wrong fix on a codebase small enough to audit in one pass; merge-conflict cost exceeds the isolation benefit. Instead:

• Every file has exactly one writing cluster, across every stage.
• Clusters needing a change elsewhere report a requirement instead of making it.
• A contended file (a global stylesheet, always) gets one owner; others submit specs.
• Verify per stage, not per cluster — a stage verifier sees all of that stage's output at once and catches cross-cluster incoherence isolated verifiers pass.
• Any fix changing URLs or routing is a serialising barrier that lands alone.

Group findings sharing a file into one unit of work regardless of priority.

Then verify the partition mechanically — do not eyeball it.

node <this-skill-dir>/tools/check-partition.mjs <baseline.json> [stages.json]

It inverts every finding's files against its cluster and fails if any file has two owners in one stage. In testing, a cluster table that stated in prose "app/globals.css has ONE owner across every stage" contained three separate files assigned to two clusters each, none flagged. A partition table reads with authority and a dispatcher will trust it, so a wrong one is worse than none.

Common Mistakes

Red Flags

• A headline score whose arithmetic isn't shown
• One deduction ledger covering several personas
• A baseline "produced" but not written to a path you can cat
• Personas invented during the run
• A probe you predicted rather than ran
• A fix list with no file column

Tools

tools/audit-link-graph.mjs produces the countable half: per-route inbound and outbound counts, clicks-from-home, orphans, and HTTP-confirmed broken links. Run with --build --verify. It needs a serving-semantics adapter per framework — the shipped one models Astro plus Vercel.