Security Audit — adversarial, evidence-grounded

A full audit, not a checklist. It runs in phases, keeps a coverage ledger so nothing is skipped silently, and — the core discipline — the agent that finds a candidate is never the one that confirms it. Severity is likelihood × impact, not deviation from a style rule. Output is machine-readable and reproducible.

Methodology inspired by Cloudflare's public security-audit-skill, rebuilt to reuse maggy's own pieces: council-review for adversarial validation, cpg-analysis (Joern/CodeQL) for static taint/data-flow, agent-teams / polyphony for isolated parallel hunters, and security for the vuln classes.

Principles (read first)

• Evidence over intuition. A finding is confirmed only when you can name the file:line of the boundary that fails and a concrete attack that crosses it.
• Finder ≠ validator. Whoever proposes a candidate must not confirm it. A different agent (or a council-review model) tries to disprove it.
• Impact-driven severity. Rate likelihood × impact, not "differs from best practice". A hardcoded key in a test fixture is not critical; an auth bypass on a tenant boundary is.
• Coverage is tracked, not assumed. Every input surface / trust boundary is a ledger unit with a status; a coverage critic hunts the gaps.
• No theater. Do not pad the report with generic "consider using HTTPS" advice. Report only boundary failures you can stand behind.

Phases

1. Reconnaissance → architecture.md + coverage-ledger.json

Map the target before hunting. Identify: entry points (HTTP routes, CLI, queue consumers, webhooks), trust boundaries (authn/authz, tenant isolation, privilege transitions), input surfaces (params, headers, files, env, deserialization), secret handling, external calls (SSRF surface), and data stores. Write a short architecture.md, then enumerate every surface as a unit in coverage-ledger.json ({id, surface, boundary, status: pending}).

2. Coverage-led hunting

For each ledger unit, hunt the relevant classes. Prefer isolated sub-agents (via agent-teams / polyphony) so one hunter's context does not bias another, and lean on cpg-analysis for data-flow/taint where a graph beats grep. Mark each unit hunted; a coverage critic pass re-reads the ledger and reopens units that were skimmed. Classes to cover (depth in security + cpg-analysis):

3. Adversarial validation (finder ≠ validator)

For each unique candidate, a different reviewer attempts to disprove it: is the tainted input actually reachable? is there a guard upstream? is the sink real? Route this through council-review (multiple models vote) for anything rated high/critical. Assign a verdict: confirmed, needs_validation, or rejected.

4. Structured output → findings.json

Emit findings.json conforming to report-schema.json (shipped in this skill). Validate it before reporting:

python3 "$(cat ~/.claude/.bootstrap-dir)/skills/security-audit/validate_findings.py" findings.json

The validator enforces the rules that keep the audit honest: unique ids, valid enums, and that every confirmed finding has a file:line location, an attack scenario, and a validated_by that differs from found_by.

5. Record verification

A fresh agent re-opens each confirmed finding and checks the cited file:line still supports the claim (code may have moved). A material mismatch drops it back to needs_validation.

6. Target-neutral report → REPORT.md

Write REPORT.md (executive summary + confirmed findings by severity), FINDINGS-DETAIL.md (per-finding evidence + remediation), and NEEDS-VALIDATION.md (candidates that could not be confirmed). Neutral tone: no vendor names, no editorializing, just boundary → attack → impact → fix.

Anti-patterns

• Confirming a finding you found yourself without an independent disprove attempt.
• Severity inflation ("uses md5" rated critical with no reachable attack).
• Grep-only hunting on a data-flow bug — use cpg-analysis.
• A report that lists advice instead of reachable, evidence-backed findings.
• Running audited target code outside a sandbox. Read and analyze; never execute untrusted target code to "see what it does".

How it fits the harness

• Slots into the base Definition of Done for security-critical changes as the "prove it's safe" gate, above the preventive security skill.
• Reuses council-review (adversarial validation), cpg-analysis (static taint), agent-teams / polyphony (isolated parallel hunters).
• Iterations are additive: a second run over the same ledger typically surfaces more, so re-run before a major release rather than trusting one pass.