Exhaustive audit framework

Honest completeness contract

A monorepo cannot fit every line into one model context. Completeness is machine-checked:

1. Inventory — every on-disk path under scope is listed (hash, bytes, line count).
2. Coverage ledger — every path reaches a terminal status (verified | finding | waived | blocked).
3. Line accountability — for each file, the auditor records that the full content was read (or a mechanical verifier covered it), with start/end line counts matching the manifest.
4. No silent skip — uncovered paths at close are a failed audit, not a soft miss.

Knowledge graph (revkg / kg_*) maps and relates; it does not replace reading file bodies. Use KG after inventory for system views and dependency edges.

When to use

Not for: tiny one-file reviews (use normal review skill); pure claim-drift (use claims validators). A Phase 0 verdict is not line-complete and must not be called exhaustive.

Artifacts (canonical locations)

Pick one run root (prefer an operator-private audits directory):

<run-root>/
  AUDIT-RUN.yml              # run metadata + scope + status
  manifest.jsonl             # one JSON object per path (machine)
  source/sha256/             # exact content snapshots for manifest hashes
  shards.json                # partition plan
  claims/                    # per-shard claim files
  ledger/
    coverage.jsonl           # path → status + auditor + evidence
    findings.jsonl           # structured findings
  reports/
    progress.md              # human progress
    assessment.md            # Phase 0 per-repo honesty (not line coverage)
    system-map.md            # optional KG-derived map
    final.md                 # closeout report

Default run root (writable operator dir; prefer fleet shared archive):

# Fleet shared cold store (not inside product git). See $REVEALFLEET_ROOT/archive/cold/README.md
export REVEALFLEET_ARCHIVE="${REVEALFLEET_ARCHIVE:-$REVEALFLEET_ROOT/archive/cold}"
export AUDIT_RUN_ROOT="${AUDIT_RUN_ROOT:-$REVEALFLEET_ARCHIVE/audits}"
RUN_ROOT="$AUDIT_RUN_ROOT/$(date -u +%Y-%m-%d)-<slug>"

Scripts (this skill)

All under $REVEALFLEET_ROOT/revskills/skills/exhaustive-audit/scripts/:

SKILL="$REVEALFLEET_ROOT/revskills/skills/exhaustive-audit"
export REVEALFLEET_ARCHIVE="${REVEALFLEET_ARCHIVE:-$REVEALFLEET_ROOT/archive/cold}"
node "$SKILL/scripts/open-run.js" --root "$REVEALFLEET_ROOT" --fleet --slug fleet-p0
# or one repo:
node "$SKILL/scripts/open-run.js" --root "$REVEALFLEET_ROOT/revealui" --slug revealui
node "$SKILL/scripts/coverage-status.js" \
  --manifest "$RUN_ROOT/manifest.jsonl" \
  --ledger "$RUN_ROOT/ledger/coverage.jsonl" \
  --mode code

Fleet --fleet walks only DEFAULT_FLEET_REPOS in scripts/lib/fleet-scope.js, including revmind and fleet-root docs/. It skips archive/, tmp/, scripts/, hidden dirs except .jv, and worktree-style names. Cold archive is opt-in: --include-archive. Override the allowlist with --repos a,b. --mode md-truth inventories *.md and *.mdx only.

Multi-session protocol

Concurrent

1. One session runs open-run (manifest + shards + AUDIT-RUN.yml).
2. Agents claim disjoint shards via claim-shard.js + workboard note (workboard.d/notes/).
3. Each session: for every path in shard → Read full file → write coverage + findings lines.
4. No two claims on same shard_id. Path-level re-claim only after release.
5. Closer session runs coverage-status.js and writes reports/final.md.

Serial

Same ledger; next session resumes first status: open or unclaimed shard. Do not rebuild manifest unless scope HEAD changed (re-hash and re-open delta only).

Coordination hardlines

• Multi-step product trees: worktrees from origin/test (not required for read-only audit of a pinned ref).
• Claims: workboard note + claims/<shard_id>.yml.
• Executable work discovered during audit → file a gap/lane; do not invent root TODO lists (canonical-work-units).
• Publish durable discoveries: kg_add_episode (agent-fact) and/or findings.jsonl.

Session workflow (every auditor)

0. Open or resume

# Open (once). Do not claim a shard in the same breath.
node "$SKILL/scripts/open-run.js" --root <SCOPE> --slug <slug> [--fleet] [--mode code]
# optional system map seed
revkg scan --repo <name>   # if POSTGRES_URL available; non-blocking

For a fleet product assessment, write reports/assessment.md (Phase 0) before claiming shards. Inventory is substrate, not the verdict.

1. Claim a shard

node "$SKILL/scripts/claim-shard.js" --run "$RUN_ROOT" --shard shard-003 --agent "$AGENT_ID"

2. Exhaustive read (per path)

For each path in the shard (from shards.json / claim file):

1. Read the entire file (paginate with offset/limit if large; cover all lines).
2. Verify line_count matches manifest (or re-count and record drift as finding). If the worktree has changed, read the manifest row's snapshot path from the run root and verify its SHA-256 before crediting the pinned content. New runs opened with open-run.js retain these snapshots with private file permissions.
3. Classify findings (see taxonomy below).
4. Append coverage record:

{
  "path": "packages/core/src/index.ts",
  "status": "verified",
  "lines_read": [1, 240],
  "manifest_lines": 240,
  "sha256": "<recomputed from the content read>",
  "agent": "grok-audit-3",
  "session": "…",
  "ts": "2026-07-22T18:00:00Z",
  "notes": ""
}

For text files, a supplied lines_read must be exactly [1, manifest.lines], with integer bounds. Empty text (lines: 0) permits omission or [0, 0]; binary manifests use lines: null. A blocked row may record a partial read. Code-mode verified rows for nonempty text require the full span.

With --check-hash, both the manifest hash and declared read hash must be nonempty strings and match. The validator uses read_sha256 when that field is present, otherwise the legacy sha256 receipt field. An invalid explicit read_sha256 cannot fall back to sha256. Compute the receipt hash from the content read; copying the manifest hash does not establish read provenance. The flag compares declared hashes and cannot independently prove that the auditor read the content.

Statuses:

Forbidden: skipped without allowlist; partial reads marked verified.

3. Finding records

Append to ledger/findings.jsonl:

{
  "id": "F-0012",
  "path": "…",
  "lines": [44, 48],
  "severity": "blocker|high|medium|low|info",
  "class": "bug|security|drift|dead|dup|doc-lie|todo|style|intent-dup",
  "title": "short",
  "body": "what is wrong; evidence",
  "gap": null,
  "agent": "…",
  "ts": "…"
}

Promotion: blocker/high → file a tracked work unit in the operator planning system (gap/lane) same or next session.

4. Knowledge graph (optional but preferred for system view)

After inventory (not instead of it):

revkg scan --repo revealui     # Tier-1 extractors
# or MCP: kg_context / kg_neighbors on package anchors

Write reports/system-map.md: major packages, edges, orphan-looking dirs (cross-check against manifest).

Publish audit conclusions:

{ "tool": "kg_add_episode", "arguments": {
  "episodeType": "agent-fact",
  "source": "exhaustive-audit",
  "content": "…",
  "nodes": [],
  "edges": []
}}

5. Complete claim + progress

# Finished the shard: mark done (not --release; release reopens it).
node "$SKILL/scripts/claim-shard.js" --run "$RUN_ROOT" --shard shard-003 --complete --agent "$AGENT_ID"
node "$SKILL/scripts/coverage-status.js" --manifest "$RUN_ROOT/manifest.jsonl" \
  --ledger "$RUN_ROOT/ledger/coverage.jsonl" --mode code \
  --write-md "$RUN_ROOT/reports/progress.md"

--release abandons a claim (status: open). --complete is terminal (status: done). Do not reclaim a done shard.

6. Close run

Only when coverage-status.js --mode <run mode> exits 0:

• reports/final.md — counts, finding index, gap ids filed, residual waivers
• reports/assessment.md — Phase 0 per-repo honesty if this was a fleet/product run
• Set AUDIT-RUN.yml status: closed
• Checkpoint / handoff fragment with run path

Finding taxonomy (minimum)

Scope defaults

Include: source, config, CI, scripts, docs, schemas, migrations, lockfiles (as units), skill/rules markdown.

Exclude by default (manifest-build --exclude-defaults):

node_modules, .git, dist, build, .next, coverage, .turbo, opensrc, .direnv, target, vendor, playwright-report, test-results, and the other names in DEFAULT_EXCLUDES inside manifest-build.js.

Fleet --fleet additionally skips (see scripts/lib/fleet-scope.js): archive/, tmp/, fleet-root scripts/, hidden dirs except .jv, worktree-style names. --include-archive is the only way to pull cold dumps into a fleet manifest.

Never exclude without recording: .env.example/templates, migrations, security gates, license files.

Verification levels (depth)

Exhaustive audit requires L2 for every path. L3 is the default for code/config; L4 for security, money, auth, and public claims surfaces.

Phase 0 — product assessment (fleet honesty)

Line inventory answers "did we account for every file." It does not answer "is this product real." Phase 0 is the honesty pass. Write reports/assessment.md from templates/assessment.md.

Per repo:

Phase 0 may finish in one session. Calling it exhaustive is a contract violation until coverage-status.js --mode code exits 0.

Subagent routing

When the active harness can select a model for each subagent, route by task class instead of using one model for every shard. For the current Codex model family, use Luna for bounded mechanical inventory, receipt consolidation, and CI status collection; Sol for disjoint source-reading shards and low-ambiguity owning fixes after the design is settled; Astra for ambiguous security design, cross-repository synthesis, and independent review of sensitive changes. Keep the independent reviewer on a separate agent/session from the author. If a task finds an unexpected trust boundary, concurrency invariant, or design choice, stop the low-cost assignment and escalate that task to Astra. Model selection never changes the coverage, evidence, or review bar.

The coordinator must pass the selected model explicitly when spawning an agent and record the requested model, task class, and actual agent/model identity in the shard report. If the harness cannot pin or report a model, say so in the receipt and keep the same review requirements; do not claim automatic model routing occurred. These names are a Codex routing profile, not fleet-wide provider requirements.

Related skills

• knowledge-graph — map/relate after inventory
• multi-agent-memory — session facts while audit runs
• security-hardening — OWASP lens on security shards
• revealui-checkpoint — close multi-session audit
• code-over-docs hardline — code wins factual conflicts

References

• references/methodology.md — full phases, concurrency rules, anti-patterns
• templates/AUDIT-RUN.yml — run metadata (prefer open-run.js)
• templates/coverage-line.json — coverage record schema
• templates/assessment.md — Phase 0 rubric
• Hardline: audit-first SDLC (studio rules)
• Work units: tracked gaps/lanes only for remediation work

Anti-patterns

MD-truth program (GAP-407)

node skills/exhaustive-audit/scripts/md-truth-check.js --self-test
# operator:
# REVEALFLEET_ARCHIVE=$REVEALFLEET_ROOT/archive/cold node …/md-truth-check.js --coverage