agentleFS
aarontzengaarontzeng/dev-lead/skills/agy-adversarial-review/SKILL.md

agy-adversarial-review

Skill3 starsChanged 8 days agoMITView on GitHub

Run an adversarial (red-team) review through the Antigravity CLI (`agy`, Gemini family). Use when the user asks for an adversarial review from Gemini/Antigravity, wants a second model family's opinion, or when another family's quota is tight and a different reviewer is needed. Pick a reviewer whose blind spots differ from the author's.

  • Deletes or force-pushes

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Before this recipe, complete the frozen-target triage step in dev-lead Phase 2.

    line 113
  2. Very specific:

    Verify the TARGET before launching — git -C "$REVIEW_TARGET_DIR" rev-parse HEAD must equal $REVIEW_HEAD (a wrong-target launch from a stale …

    line 182
  3. Distinctive:

    Keep the unguessable anchor.

    line 270

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Adversarial review via agy (Antigravity CLI)

agy is an independent path to a different model family — the reason to use it is model diversity, not convenience. A reviewer that shares the author's training shares the author's blind spots, so the value of this skill comes entirely from the reviewer being something other than the model that wrote the code.

Pairing rule across the suite: the reviewer must come from a different model family than whatever implemented the change. When quota is tight, spend the scarce model on review rather than implementation — review leverage is higher.

Before the first run of a session

Read references/agy-runtime.md (same directory). It holds the family-level mechanics shared with agy-implement — the permission allow-list this CLI needs before it can run git at all, the --add-dir workspace trap, the silent-death mode, the auth diagnosis, and the model catalogue. Every item in it was paid for with an incident; this file assumes you know them and covers only what makes a run a review.

The one-line version of the three that bite hardest: the allow-list is required or every git command is auto-denied; --add-dir is mandatory because the headless workspace root is agy's own scratch dir; and a run that returns nothing did not necessarily do nothing.

Establish an immutable review target

One frozen directory per reviewer, at the exact commit, that nothing else touches — no lead activity inside it. Not "whenever possible": a reviewer reads the WORKING TREE, not your commit. Measured — a round ran mutation testing in the same worktree mid-review and the reviewer opened a CRITICAL on a mutated, non-compiling file it was never meant to see.

Freeze it with the suite's tested helpers instead of hand-rolling the shell; every bug ever found in this step was in a hand-rolled copy. The suite-root resolver and both calls — freeze-target.sh to create it, verify-target.sh before AND after the run — are in dev-lead Phase 2, and the reasoning is in methodology.md §7. Everything below assumes $REVIEW_TARGET_DIR is that frozen directory and $REVIEW_HEAD is the SHA it was frozen at.

agy exception — the frozen target must be a self-contained CLONE, not a git worktree. Measured (2026-08-26): freeze-target.sh creates a worktree, whose .git is a one-line pointer FILE into the main repo's .git/worktrees/ — which sits outside --add-dir. agy's first read_file on that pointer is auto-denied (permission check failed for read_file ".../.git") and the whole run dies with that single line as its output (no agy version was recorded; 1.4.2 fails differently, see the note below). Every other family in this suite reads worktrees fine; agy's sandbox is the one that cannot. Freeze for agy with a local clone instead, then verify and bracket exactly as the helper flow would.

[!note] The 2026-09-15 non-reproduction, and what 2026-10-18 added.

A four-leg round on 2026-09-15 ran the agy leg against a freeze-target.sh WORKTREE (.git a 79-byte pointer file) and it completed normally, with no permission check failed and no .git read in its log. The 2026-08-26 record and that one both name no agy version, so they could not be reconciled.

On agy 1.4.2 (2026-10-18) the same probe against a worktree and a clone of one SHA settled it for that version: in the worktree every git call failed with fatal: not a git repository: <main repo>/.git/worktrees/<name>, in the clone they worked (agy-runtime.md). The lesson stands: a measurement of a CLI's behaviour needs the version it was taken on.

Freeze it with the helper's --clone mode (the freeze-target.sh call in dev-lead Phase 2 with --clone appended as a fourth argument), then verify and bracket exactly as the worktree flow does:

freeze-target.sh <repo> <sha> "$REVIEW_TARGET_DIR" --clone   # prints the SHA: REVIEW_HEAD
verify-target.sh "$REVIEW_TARGET_DIR" "$REVIEW_HEAD"          # before AND after

--clone is git clone --local --no-hardlinks --no-checkout and a detached checkout of the SHA, verified clean the same way. The clone's .git is a real directory inside --add-dir, so nothing escapes the sandbox, and no inode is shared with the source repo, so making the target read-only cannot touch it. Tear it down with rm -rf (chmod -R u+w first if it was made read-only), not git worktree remove. leg-cmd.sh agy review refuses a worktree --target.

The refusal of a polluted destination is the helper's own check on both paths (destination already exists … (refusing to touch it), rc 1) and the clone leaves an existing destination untouched; the before/after bracket proves the target did not CHANGE, not that it is the one this round created.

Run it

Since 0.6.30 the brief is a LENS inside this leg's framing (references/adversarial-framing.md), and the leg reads material it cannot fetch itself. agy in plan mode on a locked-down machine runs no git, so the builder materializes the range into $RUN_DIR/evidence -- DIFF.patch, base/<path> for every changed path that exists at the base, BLOBS.txt (mode and blob id on both sides), COMMITS.txt -- read-only, outside the frozen target, and a second --add-dir grants it. The frame also sets a 15-minute budget and caps the findings at 8: long briefs that pointed agy at many files ran into --print-timeout and returned partial output (runtime file). Adopted on measurement: three frozen targets with known answers, three runs per arm, the plain brief 3 and this frame 8 of the known defects, at a small false-positive cost (runtime file, calibration table).

Before this recipe, complete the frozen-target triage step in dev-lead Phase 2; the review launcher requires its record for this exact commit.

RUN_DIR=$(mktemp -d "${TMPDIR:-/tmp}/agy-review.XXXXXX")   # fresh per leg
# Write the lens to "$RUN_DIR/prompt.md" in its own step.
# The roster names the model for this leg (roster.py plan); this is an
# example spelling. Confirm it is in the account catalogue; if not, pick an
# available suffix variant.
AGY_MODEL=gemini-3.8-flash-high
BASE=$(git -C "$REVIEW_TARGET_DIR" merge-base origin/main HEAD)   # the span: see below
DEV_LEAD=${DEV_LEAD_ROOT:-$(ls -d "$HOME"/.claude/plugins/cache/dev-lead/dev-lead/* 2>/dev/null | sort -V | tail -1)}
eval "$("$DEV_LEAD/scripts/leg-cmd.sh" agy review --model "$AGY_MODEL" \
  --base "$BASE" --target "$REVIEW_TARGET_DIR" --run-dir "$RUN_DIR")" > "$RUN_DIR/review.log" 2>&1

What it runs, spelled out (for a lead without the script):

python3 "$DEV_LEAD/scripts/review-prompt.py" --adapter agy --base "$BASE" \
  --target "$REVIEW_TARGET_DIR" --evidence "$RUN_DIR/evidence" \
  < "$RUN_DIR/prompt.md" > "$RUN_DIR/framed-prompt.md" && \
agy -p "$(cat "$RUN_DIR/framed-prompt.md")" \
    --model "$AGY_MODEL" \
    --mode plan \
    --sandbox \
    --add-dir "$REVIEW_TARGET_DIR" \
    --add-dir "$RUN_DIR/evidence" \
    --print-timeout 20m0s

The builder refuses an evidence directory that already has content -- give every leg a fresh $RUN_DIR -- and one inside the frozen target. The evidence is read-only for the whole run: chmod -R u+w "$RUN_DIR/evidence" before you clean it up.

--disable-slash-commands is deliberately absent here, and adding it back silently removes plan mode. The CLI says so — warning: --mode plan has no effect while slash command expansion is disabled — and measured, with both flags the delegate created a file it was asked to create; with --mode plan alone it refused and returned a plan instead. The pair leaves a review run with NO behavioral no-write layer at all while the prompt and this file still claim one. Keep paths in the prompt from starting a line with / instead; the implement role keeps the flag (runtime file: accept-edits survives it).

Role-specific choices in that command:

  • --mode plan — research and report, never edit. It is a behavioral mode, not a security boundary, so the no-write intent is stated again in the prompt and verified in the repo afterward. It is also the layer --disable-slash-commands silently cancels (above) — never pass both.
  • --print-timeout 20m0s — a framed review takes 3–7 minutes (measured 2026-09-24), a real review of a few files 5–10, and the CLI default (5m0s) cuts it off mid-flight. 20 leaves room above the frame's 15-minute budget; a run that hits the timeout returns PARTIAL output, which is not a review.
  • Gemini model, effort omitted — AGY_MODEL above is an EXAMPLE spelling; the model is whatever the roster names for this leg (the maintainer's example roster says gemini-3.8-flash-medium), and only when the account catalogue offers it. The -high suffix IS the effort; do not pass --effort (measured: omitting it works, and a mismatched value is a hard CLI error, so the flag can only break the run or restate the suffix). Verify from the log which model actually served; a model with no calibration rows yet is not a gate merely because it is newer (the runtime file has 3.8's rows).
  • Model family — the CLI may expose a second family's pool (e.g. Claude) on separate quota, a legitimate cross-family reviewer for anything that family did NOT write. --effort stays absent there too — it is rejected outright for Claude models (see runtime).

Verify the TARGET before launching — git -C "$REVIEW_TARGET_DIR" rev-parse HEAD must equal $REVIEW_HEAD (a wrong-target launch from a stale cwd is measured in the codex twin of this skill; here the same mistake is a wrong --add-dir, which is why the check names the directory explicitly). Because plan mode is behavioral rather than enforced, bracket the run:

git -C "$REVIEW_TARGET_DIR" rev-parse HEAD        # before, and again after —
git -C "$REVIEW_TARGET_DIR" status --porcelain=v1 # -C, or you bracket the
                                                  # lead's own cwd, not the target

Running this leg as a subagent? You are a leaf — block, do not "wait". agy -p is a foreground CLI, so handing it to the host's background mechanism and then ending your turn on "waiting for the notification" abandons the run — nothing will wake you. Poll to exit inside a single tool call, and issue another immediately if it times out (dev-lead Phase 2).

Verifying the target is not verifying the SPAN. The check above proves you launched at the right commit; it says nothing about which range the prompt tells the delegate to read. For a topic branch that range is the merge-base — never the target branch name — and the two spellings look identical but are not: git log A..B is "commits in B not in A" (what you want), but git diff A..B means git diff A B, not merge-base. Measured: a prompt naming a 2-commit chain but reading it with a two-dot diff, while the target was 10 commits past the branch point, handed the delegate two other teams' files as part of the author's change — nothing errored, both HEAD assertions passed, and the contamination surfaced only when findings were checked file-by-file against the real diff.

Pin the base once and hand the delegate that, or use three dots:

BASE=$(git merge-base origin/main HEAD)
git diff --stat "$BASE" HEAD          # file list must match the change under review

Any changed HEAD or unexpected dirty-state delta is a failed safety check — preserve the frozen target and inspect before trusting or relaying a single line of the review output.

Writing the prompt

The same discipline as any red-team review, and it matters more than the model:

  • Tell it to falsify, not to confirm — but frame it as first-party work. Open with "this is a routine pre-merge correctness review of our own code by the team that wrote it", then "for each property below, say whether it HOLDS or is BROKEN, and if broken, exactly how." Avoid "RED TEAM" and "attack" as the opening register — see the refusal gotcha below.
  • List the claims explicitly. Quote the properties the code says it has, and ask for each by name. Vague "review this" gets vague output; a numbered claim list gets numbered rebuttals.
  • Name the attack surface you already worry about — crash points, partial writes, integer overflow, concurrent paths. It will find more, but not if the prompt reads like a code-style request.
  • Demand a trigger and a file:line per finding, and ask it to say in one line which claims survived.
  • State the review-only intent in the prompt as well as the flags, and forbid recursive CLI invocation.
  • Forbid running tests/builds, and give the impulse somewhere to go: "if confirming a finding requires execution, report it as a finding that names the exact command and the result that would confirm it — we run it." Note the runtime's honest caveat: if your allow-list contains a test runner (globally — there is no per-role scoping), a disobedient reviewer can actually execute it, while every other build command dies silently with zero output. Both outcomes are wrong; the prompt line stays mandatory.
  • A brief may only name commands on this machine's allow-list, and a pipeline is only as permitted as its least-permitted stage. The runtime file's one-time permissions.allow setup is a prerequisite: with it the delegate runs git show/log/diff/status/branch/rev-parse; without it even read-only git log dies with zero output. What bites after that setup is the pipeline — git show … | sed … dies on sed, silently, on a machine where git show is allowed. When the material needs a tool that is not on the list, materialize it yourself rather than widening the list for one run: references/materializing-evidence.md carries the four properties that step must have and a worked example.
  • Ask what the tests do not enumerate, in those words (see docs/methodology.md — measured as the highest-yield sentence in the prompt across every family).
  • Forbid praise explicitly. Without it the default register drifts to summarizing and complimenting.
  • Size the evidence gate to what YOU cannot check — and on a DOCUMENT (ADR, spec, contract) that is a different list than on code. Demanding re-derivation of facts the author already verified turns a document read into a repo sweep that blows the print timeout with zero output (measured). Keep the unguessable anchor; ask instead what is stated-but-unverifiable, what a reader stopping mid-document would wrongly conclude, and which claims are the author's own inference.
  • This leg's measured strength is ATTACK PATHS — brief it on sequences. In paired rounds against a free-pool leg on the same targets, this leg found the bearer credential an intermediary defeats, the single-slot race, and the rate-limit pre-check that bounded nothing — each with a concrete sequence — while the other leg passed all three. The mirror also held: the other leg caught document-consistency defects this one passed. Phrase properties for this leg as "an attacker does X, walk it" and route "does this contradict document Y" to a consistency-strong leg. (Your own calibration journal may differ — measure.)
  • Give every claim an evidence gate with an unguessable anchor. This family is where the motivating incident happened: a format-compliant all-HOLDS verdict in three minutes, having cited nothing. Require per file its line count AND the verbatim last line; per claim a file:line plus the quoted code; state that NOT REACHED is acceptable and HOLDS-without-quote is not.

Reporting back

  • Relay findings faithfully — do not soften or drop them.
  • Verify every finding against the code yourself before acting. It is confidently wrong sometimes: in the run that motivated this skill, six of eight findings were real (two severe and genuinely subtle), one was wrong on its own premise, one was real but astronomically unreachable. Say plainly which you confirmed, which you reject, and why — and hold a rejection to the same evidence standard the finding was held to.
  • Grep every quote before you use it. The frame tells the leg to quote only text it opened, but measured the same day it was adopted: a framed leg cited lines 131-189 of a 92-line file and quoted requirement text that exists nowhere, on a documentation packet, while its conclusions were partly right; on a code packet its quotes were exact but it put a test in the wrong change and called a present build line missing. A quote that is not in the file at the cited lines voids that finding; an attribution is checked against COMMITS.txt and DIFF.patch, and an absence claim against the files, before either is relayed.
  • Review-only: do not apply fixes in the same breath unless asked.

The refusal gotcha

Red-team framing can trip a refusal. "You are a RED TEAM reviewer… falsify claims" over safety-critical code (control systems, medical) has come back as a refusal — the model reads it as an attack on a third party. Reframe as what it actually is: a pre-merge correctness review of first-party code by its own team, with HOLDS/BROKEN verdicts per property. The findings are the same; only the framing changes.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.