agentleFS
darkroomengineeringdarkroomengineering/cc-settings/skills/poke-holes/SKILL.md

poke-holes

Skill52 starsChanged 50 days agoMITView on GitHub

Adversarial verification with three competing agents (issue-finder, disprover, judge). Triggers "double check", "are you sure", "poke holes", or a critical fix before production.

The file

We split this skill into the rules it gives an agent

8 rules. About 0 are common, 6 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Create each fresh panelist with spawn_agent, deliver context while it runs with send_message, trigger another turn once it is idle with …

    line 11
  2. Distinctive:

    Only read-only reviewers may overlap.

    line 17
  3. Distinctive:

    Use three separate native agents instead of the Claude bridge: spawn a fresh issue-finder, wait until it finishes, pass its report to a …

    line 47
  4. Distinctive:

    Follow the lifecycle above for each role.

    line 50
  5. Distinctive:

    All three agents here are Claude — one model family with shared blind spots, which is the exact self-preferential bias this skill exists to …

    line 55
  6. Very specific:

    If the codex-verifier spawn fails, or it reports that Bash was stripped (forked skill contexts), run bun …

    line 63
  7. Distinctive:

    If you catch yourself thinking any of the following, STOP — you are rationalizing skipping verification:

    line 174
  8. Distinctive:

    If any agent (including yourself) uses these phrases, verification is NOT complete — restart the verification step:

    line 187

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Adversarial Verification

Standalone Codex

Create each fresh panelist with spawn_agent, deliver context while it runs with send_message, trigger another turn once it is idle with followup_task, wait with wait_agent, and stop its current turn with interrupt_agent only when necessary. Never spawn codex-verifier and never run codex-run.ts from inside Codex.

Only read-only reviewers may overlap. Writers share the working tree unless the live host explicitly offers isolation, so any implementer and test-writer follow-up must use non-overlapping ownership and run serially.

When to Use

  • Security-sensitive code (auth, crypto, permissions)
  • Data integrity (migrations, schema changes, ETL)
  • Financial logic (payments, billing, calculations)
  • Breaking changes (API contracts, public interfaces)

The Three-Agent Pattern

Agent 1: Finder

Agent(reviewer, "You are a bug finder. Analyze the following code/changes thoroughly.
Score yourself: +1 for low-impact issues, +5 for medium-impact, +10 for critical.
Report every potential issue you find — edge cases, race conditions, missing validation,
security holes, logic errors, performance problems.
Report your total score at the end.

Target: [describe what to verify]
Files: [list files]")

Finder over-reports by design — this is the superset of all possible issues.

Standalone Codex panel

Use three separate native agents instead of the Claude bridge: spawn a fresh issue-finder, wait until it finishes, pass its report to a separate disprover, then wait until that agent finishes before spawning the judge with both reports. Follow the lifecycle above for each role. Skip the Claude cross-model finder below.

Cross-model finder (when the Codex bridge is available)

All three agents here are Claude — one model family with shared blind spots, which is the exact self-preferential bias this skill exists to fight. When the Codex bridge is available, add a non-Claude voice to the panel by running, in parallel with Agent 1:

Agent(codex-verifier, "Independently find issues in the current diff. Report findings by severity.")

Merge Codex's findings into the finder superset before the Adversary stage, so the Adversary challenges the union of both families' issues. (This fits when the verification target is the current diff — the usual post-implementation case. For arbitrary non-diff code, fall back to the all-Claude panel.) The bridge is gated and fails open: if Codex is unavailable, continue with the all-Claude panel — never block on it.

If the codex-verifier spawn fails, or it reports that Bash was stripped (forked skill contexts), run bun "$HOME/.claude/src/scripts/codex-run.ts" review directly instead — never skip the cross-model pass.

Agent 2: Adversary

Takes the finder's output and tries to disprove each issue.

Agent(reviewer, "You are an adversarial reviewer. For each issue below, try to DISPROVE it.
Score yourself: +points of the bug for each you successfully disprove,
but -2x the points if you wrongly disprove a real issue.

Issues to challenge:
[paste finder output]

For each issue, state:
- DISPROVED: [reason it's not actually an issue]
- CONFIRMED: [reason it is a real issue]
- UNCERTAIN: [what would need to be checked]")

Adversary filters aggressively but cautiously — this is the subset of likely-real issues.

Find the one safety fact

Most risky-looking changes are safe because of one fact ("this call only drops already-dead cache entries"). Before the Adversary runs, name that fact and record how far its evidence got:

  1. Asserted: worthless on its own.
  2. Cited: a real file:line, or the dependency's own source at its pinned version.
  3. Walked: the failure traced step by step, and it does not reach.
  4. Ran: a script or test that calls the real code and fails loud if the fact is wrong.
  5. Reproduced: shown in the running app.

Push the fact to level 4 when that is cheap, usually one small script importing the same library the app ships. If it holds, it clears most finder issues at once; hand it to the Adversary as evidence. If you cannot get past level 2, say so in the report. Look past what grep shows: the dependency's source and local patches, teardown and microtask timing, JSON or DB columns, wire formats, feature flags, and code several hops downstream.

Agent 3: Referee

Takes both inputs and produces the final verdict.

Agent(explore, "You are a neutral referee scoring two reviewers.
You will get +1 for each correct judgment and -1 for each incorrect one.
The ground truth exists and will be checked against your answers.

For each issue, produce a final verdict:

REAL BUG — with severity (Critical/Warning/Minor)
FALSE POSITIVE — explain why
NEEDS HUMAN CHECK — genuinely ambiguous

Finder report:
[paste finder output]

Adversary report:
[paste adversary output]")

Workflow

  1. Identify scope — what code/changes need verification
  2. Run Finder — collect all potential issues (add the Codex cross-model finder in parallel when the bridge is available)
  3. Run Adversary — challenge finder's output
  4. Run Referee — judge both outputs
  5. Report — present final verdicts in the Output Format below, including its Verdict line

Sequential — each agent depends on the previous output. However small the target, run at least Lightweight Mode's two agents; reading the code yourself is not a substitute for the panel.

Output Format

## Adversarial Verification Report

### Scope
[What was verified]

### Verdict: [PASS / FAIL / NEEDS REVIEW]

### Safety Fact
[The one fact the change is safe because of] — evidence level [1-5], proof: [script output, file:line, or "unproven"]

### Confirmed Issues
| # | Severity | Issue | File:Line | Action Required |
|---|----------|-------|-----------|----------------|
| 1 | Critical | [description] | [location] | [what to fix] |

### Disproved (False Positives)
| # | Claimed Issue | Why Not Real |
|---|---------------|--------------|
| 1 | [description] | [reason] |

### Needs Human Check
| # | Issue | Why Ambiguous |
|---|-------|---------------|
| 1 | [description] | [what to check] |

### Confidence
Finder: N issues. Adversary disproved: M. Referee confirmed: K.

Lightweight Mode

For smaller changes, skip the referee:

Agent(reviewer, "Find all issues in [target]. Be thorough.")
Agent(reviewer, "Challenge each issue: [paste output]. Disprove what you can.")

Review surviving issues yourself.

Rationalization Counters

If you catch yourself thinking any of the following, STOP — you are rationalizing skipping verification:

RationalizationWhy It's Wrong
"The change is too simple to need three agents"Simple changes to auth/payments have caused the worst production incidents
"I already reviewed it myself"Self-review has a known blind spot for logic errors you just wrote
"It's just a refactor, behavior doesn't change"Refactors that "don't change behavior" are the #1 source of subtle regressions
"Tests are passing, that's enough"Tests verify expected behavior; adversarial review finds unexpected behavior
"This would take too long"A 5-minute verification is cheaper than a production incident
"The reviewer agent already checked it"The reviewer checks quality; verification checks correctness under adversarial pressure

Red-Flag Phrases

If any agent (including yourself) uses these phrases, verification is NOT complete — restart the verification step:

  • "should work" / "should be fine"
  • "probably" / "likely" / "most likely"
  • "I believe" / "I think" (without evidence)
  • "Done!" / "All good!" / "Looks great!"
  • "I don't see any issues"
  • "This is straightforward"

Each of these must be replaced with evidence: a specific test, a concrete trace through the code, or a cited invariant.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.