agentleFS
petekppetekp/claude-code-setup/skills/exhaustive-systems-analysis/SKILL.md

exhaustive-systems-analysis

Skill48 starsChanged 51 days agoMITView on GitHub

Perform evidence-driven, multi-subsystem audits of real codebases to find correctness bugs, race conditions, security gaps, stale documentation, dead code, and production-readiness risks. Use when asked to audit a system end-to-end, verify agent-written code before shipping, analyze a subsystem for correctness across multiple modules, or produce a structured risk report for a real implementation. Prefer other skills for a single isolated bug, a proposal or document review, or a dedicated dead-code cleanup.

The file

Show its rules and text

We split this skill into the rules it gives an agent

22 rules. About 0 are common, 18 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈18

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill for full-system correctness work.

    line 9
  2. Very specific:

    Default to chat-first output.

    line 20
  3. Distinctive:

    Do not start fixing code while auditing unless the user explicitly asks for fixes.

    line 22
  4. Very specific:

    Before reading deeply, write a one-screen scope brief using the template in references/templates.md.

    line 28
  5. Distinctive:

    If the request is broad, narrow it to the modules that can actually change user outcomes or ship readiness.

    line 37
  6. Distinctive:

    Read only the materials that establish intended behavior:

    line 41
  7. Distinctive:

    Update or discard hypotheses as evidence comes in.

    line 78
  8. Distinctive:

    Read the subsystem end-to-end:

    line 82
  9. Distinctive:

    start at entrypoints and trace the happy path

    line 83
  10. Distinctive:

    run targeted searches, commands, or tests when they strengthen the evidence

    line 86
  11. Distinctive:

    When subagents are available, assign one bounded subsystem per subagent with disjoint files and ask for:

    line 91
  12. Distinctive:

    Every finding must separate observation from inference.

    line 99
  13. Very specific:

    Use Confirmed only when the bug is directly demonstrated by code, a failing test, a repro path, or a hard contradiction.

    line 114
  14. Distinctive:

    run a final residue sweep for stale docs, deprecated names, orphaned helpers, temp flags, TODO or FIXME clusters, and risky APIs

    line 121
  15. Distinctive:

    Prefer stronger evidence over more words.

    line 127
  16. Distinctive:

    Prefer user-impacting correctness issues over stylistic cleanup.

    line 143
  17. Distinctive:

    Keep related findings separate unless they share the same root cause.

    line 144
  18. Distinctive:

    If nothing serious is wrong, say so directly and still report residual risk and unverified surfaces.

    line 145
  19. Distinctive:

    Do not write "looks wrong" or "might be an issue" without saying what you checked and what would prove or disprove it.

    line 146
  20. Very specific:

    Use the templates in references/templates.md for:

    line 148
  21. Distinctive:

    Use single-session mode for small audits.

    line 157
  22. Distinctive:

    every reported finding has evidence, confidence, and a concrete location

    line 183

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Exhaustive Systems Analysis

Use this skill for full-system correctness work. The job is to map the system, identify the highest-risk behaviors, prove or refute concrete failure hypotheses, and leave behind a report another engineer can act on without re-reading the whole codebase.

Failure Modes To Prevent

  1. Surface-level audits that scan files without following behavior end-to-end
  2. False certainty: reporting suspicions as bugs without enough evidence
  3. Context drift across large audits with many subsystems
  4. Cosmetic reviews that miss the real correctness and ship-readiness risks

Operating Mode

  • Default to chat-first output. Return findings inline unless the user asks for docs or the audit clearly needs multi-session artifacts.
  • Switch to artifact mode for large or resumable audits. Use docs/audit/ or .claude/docs/audit/, matching the repo's existing conventions.
  • Do not start fixing code while auditing unless the user explicitly asks for fixes. This skill is for diagnosis, proof, and prioritization.

Workflow

0. Calibrate The Audit

Before reading deeply, write a one-screen scope brief using the template in references/templates.md.

Capture:

  • system or area under review
  • user-visible workflows or contracts that matter most
  • likely high-risk surfaces: state, side effects, concurrency, auth, persistence, external integrations
  • out-of-scope areas
  • output mode: chat-first or artifact mode

If the request is broad, narrow it to the modules that can actually change user outcomes or ship readiness.

1. Load Intent Before Code

Read only the materials that establish intended behavior:

  • README, CLAUDE.md, architecture docs, ADRs
  • tests that describe user-visible or contract behavior
  • recent commits touching the target area
  • TODO, FIXME, HACK, and "known issues"
  • incident notes, bug reports, or issue tracker items if available

Extract:

  • critical workflows
  • external surfaces
  • hotspots and recent churn
  • manual-only surfaces that cannot be fully verified from code alone

2. Build The Coverage Ledger

Map the system into subsystems before deep analysis. Use the coverage ledger template in references/templates.md.

For each subsystem record:

  • name
  • entrypoints
  • files or directories in scope
  • invariants or promised behaviors
  • side effects
  • risk level
  • status: planned | in_progress | done | follow_up

Prioritize by user impact first, then by side effects, concurrency, privilege, and recent churn. Folder structure alone is not a priority system.

3. Generate Hypotheses Before The Deep Pass

For each high- or medium-risk subsystem, write 2-3 concrete hypotheses before diving in. Good hypotheses are falsifiable and tied to a behavior boundary.

Examples:

  • "A failure between write A and write B can leave persisted state inconsistent."
  • "The retry path duplicates a side effect because idempotence is not enforced."
  • "The docs promise behavior X, but the implementation falls through to Y on invalid input."

Update or discard hypotheses as evidence comes in. This step prevents aimless scanning.

4. Audit One Subsystem At A Time

Read the subsystem end-to-end:

  • start at entrypoints and trace the happy path
  • trace error paths, cleanup paths, cancellation or shutdown, and retries
  • compare implementation to tests, docs, types, and public contracts
  • run targeted searches, commands, or tests when they strengthen the evidence
  • record exact commands, searches, and scopes when they support a finding

Select only the relevant checklist sections from references/checklists.md. Do not load every checklist if the subsystem only needs one or two.

When subagents are available, assign one bounded subsystem per subagent with disjoint files and ask for:

  • hypotheses checked
  • findings with exact citations
  • coverage gaps
  • suggested next verification step

5. Classify Findings With Evidence, Status, And Confidence

Every finding must separate observation from inference.

Required fields:

  • Severity: Critical | High | Medium | Low
  • Status: Confirmed | Likely | Needs follow-up
  • Confidence: High | Medium | Low
  • Type: Bug | Race condition | Security | Stale docs | Dead code | Design flaw | Reliability
  • Location: exact file path and line or function
  • Impacted behavior: the user-visible workflow, invariant, or contract at risk
  • Observed evidence: code citation, command output, test result, log, or search result
  • Inference: why that evidence implies the reported problem
  • What I checked: searches, tests, docs, commits, or alternate explanations ruled out
  • Recommendation: the smallest credible next action
  • Next verification step: required when status is Needs follow-up

Use Confirmed only when the bug is directly demonstrated by code, a failing test, a repro path, or a hard contradiction. Use Likely when the reasoning is strong but not directly reproduced. Use Needs follow-up when something is suspicious but the evidence is incomplete.

6. Run A Convergence Pass

After subsystem reviews:

  • deduplicate cross-cutting findings
  • re-rank by severity and user impact
  • run a final residue sweep for stale docs, deprecated names, orphaned helpers, temp flags, TODO or FIXME clusters, and risky APIs
  • record exact residue queries and counts if they matter to the conclusion
  • list coverage gaps explicitly instead of pretending the audit was complete where it was not

Evidence Standard

Prefer stronger evidence over more words. From strongest to weakest:

  1. failing or targeted test
  2. reproducible path with exact steps
  3. direct code contradiction with exact citations
  4. logs, telemetry, or command output
  5. scoped search results with counts
  6. static reasoning

Static reasoning alone can still be valuable, but it should usually produce Likely, not Confirmed.

For dead code or stale docs, always show what you searched and why you believe the code or documentation is obsolete. A dead-code claim without a consumer search is incomplete.

Reporting Rules

  • Lead with findings, not the methodology recap.
  • Prefer user-impacting correctness issues over stylistic cleanup.
  • Keep related findings separate unless they share the same root cause.
  • If nothing serious is wrong, say so directly and still report residual risk and unverified surfaces.
  • Do not write "looks wrong" or "might be an issue" without saying what you checked and what would prove or disprove it.

Use the templates in references/templates.md for:

  • scope brief
  • coverage ledger
  • finding format
  • chat-first summary
  • artifact-mode audit directory

Session Management

Use single-session mode for small audits. For large audits or when context is tight, create a lightweight control plane:

  • 00-plan.md for the scope brief and coverage ledger
  • one file per subsystem only if the audit is large enough to justify it
  • SUMMARY.md for consolidated findings and fix order
  • HANDOFF.md if work will continue later

A good handoff includes:

  • what was covered
  • what is now believed to be true
  • what remains unverified
  • current blockers
  • exact next steps

Anti-Patterns

  • scanning directories without identifying entrypoints or invariants
  • reporting every code smell as a finding
  • calling something dead code without a consumer search
  • calling something a bug without showing the broken behavior or violated contract
  • collapsing multiple subsystems into one giant writeup
  • hiding uncertainty instead of marking Needs follow-up

Completion Criteria

The audit is complete when:

  1. high-risk subsystems have a ledger entry and a final status
  2. every reported finding has evidence, confidence, and a concrete location
  3. the final report includes fix order plus unverified surfaces
  4. cross-cutting and residue findings have been consolidated
  5. the report is honest about what was not proven

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.