agy-adversarial-review
Skill3 starsChanged 8 days agoMITView on GitHub
Run an adversarial (red-team) review through the Antigravity CLI (`agy`, Gemini family). Use when the user asks for an adversarial review from Gemini/Antigravity, wants a second model family's opinion, or when another family's quota is tight and a different reviewer is needed. Pick a reviewer whose blind spots differ from the author's.
- Deletes or force-pushes
In this repository15 other files it gives its agents.
- templates/AGENTS.mdAGENTS.md
- agy-implementSkill
- claude-adversarial-reviewSkill
- claude-implementSkill
- codex-adversarial-reviewSkill
- codex-implementSkill
- configSkill
- cursor-adversarial-reviewSkill
- cursor-implementSkill
- dev-leadSkill
- grok-adversarial-reviewSkill
- grok-implementSkill
- opencode-adversarial-reviewSkill
- opencode-implementSkill
- quota-setupSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 113
Before this recipe, complete the frozen-target triage step in dev-lead Phase 2.
- Very specific: line 182
Verify the TARGET before launching — git -C "$REVIEW_TARGET_DIR" rev-parse HEAD must equal $REVIEW_HEAD (a wrong-target launch from a stale …
- Distinctive: line 270
Keep the unguessable anchor.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Adversarial review via agy (Antigravity CLI)
agy is an independent path to a different model family — the reason to use
it is model diversity, not convenience. A reviewer that shares the
author's training shares the author's blind spots, so the value of this skill
comes entirely from the reviewer being something other than the model that
wrote the code.
Pairing rule across the suite: the reviewer must come from a different model family than whatever implemented the change. When quota is tight, spend the scarce model on review rather than implementation — review leverage is higher.
Before the first run of a session
Read references/agy-runtime.md (same
directory). It holds the
family-level mechanics shared with agy-implement — the permission
allow-list this CLI needs before it can run git at all, the --add-dir
workspace trap, the silent-death mode, the auth diagnosis, and the model
catalogue. Every item in it was paid for with an incident; this file assumes
you know them and covers only what makes a run a review.
The one-line version of the three that bite hardest: the allow-list is
required or every git command is auto-denied; --add-dir is mandatory
because the headless workspace root is agy's own scratch dir; and a run that
returns nothing did not necessarily do nothing.
Establish an immutable review target
One frozen directory per reviewer, at the exact commit, that nothing else touches — no lead activity inside it. Not "whenever possible": a reviewer reads the WORKING TREE, not your commit. Measured — a round ran mutation testing in the same worktree mid-review and the reviewer opened a CRITICAL on a mutated, non-compiling file it was never meant to see.
Freeze it with the suite's tested helpers instead of hand-rolling the shell;
every bug ever found in this step was in a hand-rolled copy. The suite-root
resolver and both calls — freeze-target.sh to create it, verify-target.sh
before AND after the run — are in
dev-lead Phase 2, and the reasoning is in
methodology.md §7. Everything below assumes
$REVIEW_TARGET_DIR is that frozen directory and $REVIEW_HEAD is the SHA it
was frozen at.
agy exception — the frozen target must be a self-contained CLONE, not a
git worktree. Measured (2026-08-26): freeze-target.sh creates a worktree,
whose .git is a one-line pointer FILE into the main repo's .git/worktrees/
— which sits outside --add-dir. agy's first read_file on that pointer is
auto-denied (permission check failed for read_file ".../.git") and the whole
run dies with that single line as its output (no agy version was recorded; 1.4.2
fails differently, see the note below). Every other family in this
suite reads worktrees fine; agy's sandbox is the one that cannot. Freeze for
agy with a local clone instead, then verify and bracket exactly as the helper
flow would.
[!note] The 2026-09-15 non-reproduction, and what 2026-10-18 added.
A four-leg round on 2026-09-15 ran the agy leg against a
freeze-target.shWORKTREE (.gita 79-byte pointer file) and it completed normally, with nopermission check failedand no.gitread in its log. The 2026-08-26 record and that one both name no agy version, so they could not be reconciled.On agy 1.4.2 (2026-10-18) the same probe against a worktree and a clone of one SHA settled it for that version: in the worktree every
gitcall failed withfatal: not a git repository: <main repo>/.git/worktrees/<name>, in the clone they worked (agy-runtime.md). The lesson stands: a measurement of a CLI's behaviour needs the version it was taken on.
Freeze it with the helper's --clone mode (the freeze-target.sh call in
dev-lead Phase 2 with --clone appended as a fourth
argument), then verify and bracket exactly as the worktree flow does:
freeze-target.sh <repo> <sha> "$REVIEW_TARGET_DIR" --clone # prints the SHA: REVIEW_HEAD
verify-target.sh "$REVIEW_TARGET_DIR" "$REVIEW_HEAD" # before AND after--clone is git clone --local --no-hardlinks --no-checkout and a detached
checkout of the SHA, verified clean the same way. The clone's .git is a real
directory inside --add-dir, so nothing escapes the sandbox, and no inode is
shared with the source repo, so making the target read-only cannot touch it.
Tear it down with rm -rf (chmod -R u+w first if it was made read-only), not
git worktree remove. leg-cmd.sh agy review refuses a worktree --target.
The refusal of a polluted destination is the helper's own check on both paths
(destination already exists … (refusing to touch it), rc 1) and the clone
leaves an existing destination untouched; the before/after bracket proves the
target did not CHANGE, not that it is the one this round created.
Run it
Since 0.6.30 the brief is a LENS inside this leg's framing
(references/adversarial-framing.md),
and the leg reads material it cannot fetch itself. agy in plan mode on a
locked-down machine runs no git, so the builder materializes the range into
$RUN_DIR/evidence -- DIFF.patch, base/<path> for every changed path that
exists at the base, BLOBS.txt (mode and blob id on both sides), COMMITS.txt
-- read-only, outside the frozen target, and a second --add-dir grants it.
The frame also sets a 15-minute budget and caps the findings at 8: long briefs
that pointed agy at many files ran into --print-timeout and returned partial
output (runtime file). Adopted on measurement: three frozen targets with known
answers, three runs per arm, the plain brief 3 and this frame 8 of the known
defects, at a small false-positive cost (runtime file, calibration table).
Before this recipe, complete the frozen-target triage step in dev-lead Phase 2; the review launcher requires its record for this exact commit.
RUN_DIR=$(mktemp -d "${TMPDIR:-/tmp}/agy-review.XXXXXX") # fresh per leg
# Write the lens to "$RUN_DIR/prompt.md" in its own step.
# The roster names the model for this leg (roster.py plan); this is an
# example spelling. Confirm it is in the account catalogue; if not, pick an
# available suffix variant.
AGY_MODEL=gemini-3.8-flash-high
BASE=$(git -C "$REVIEW_TARGET_DIR" merge-base origin/main HEAD) # the span: see below
DEV_LEAD=${DEV_LEAD_ROOT:-$(ls -d "$HOME"/.claude/plugins/cache/dev-lead/dev-lead/* 2>/dev/null | sort -V | tail -1)}
eval "$("$DEV_LEAD/scripts/leg-cmd.sh" agy review --model "$AGY_MODEL" \
--base "$BASE" --target "$REVIEW_TARGET_DIR" --run-dir "$RUN_DIR")" > "$RUN_DIR/review.log" 2>&1What it runs, spelled out (for a lead without the script):
python3 "$DEV_LEAD/scripts/review-prompt.py" --adapter agy --base "$BASE" \
--target "$REVIEW_TARGET_DIR" --evidence "$RUN_DIR/evidence" \
< "$RUN_DIR/prompt.md" > "$RUN_DIR/framed-prompt.md" && \
agy -p "$(cat "$RUN_DIR/framed-prompt.md")" \
--model "$AGY_MODEL" \
--mode plan \
--sandbox \
--add-dir "$REVIEW_TARGET_DIR" \
--add-dir "$RUN_DIR/evidence" \
--print-timeout 20m0sThe builder refuses an evidence directory that already has content -- give
every leg a fresh $RUN_DIR -- and one inside the frozen target. The evidence
is read-only for the whole run: chmod -R u+w "$RUN_DIR/evidence" before you
clean it up.
--disable-slash-commands is deliberately absent here, and adding it back
silently removes plan mode. The CLI says so — warning: --mode plan has no
effect while slash command expansion is disabled — and measured, with both
flags the delegate created a file it was asked to create; with --mode plan
alone it refused and returned a plan instead. The pair leaves a review run
with NO behavioral no-write layer at all while the prompt and this file still
claim one. Keep paths in the prompt from starting a line with / instead;
the implement role keeps the flag (runtime file: accept-edits survives it).
Role-specific choices in that command:
--mode plan— research and report, never edit. It is a behavioral mode, not a security boundary, so the no-write intent is stated again in the prompt and verified in the repo afterward. It is also the layer--disable-slash-commandssilently cancels (above) — never pass both.--print-timeout 20m0s— a framed review takes 3–7 minutes (measured 2026-09-24), a real review of a few files 5–10, and the CLI default (5m0s) cuts it off mid-flight. 20 leaves room above the frame's 15-minute budget; a run that hits the timeout returns PARTIAL output, which is not a review.- Gemini model, effort omitted —
AGY_MODELabove is an EXAMPLE spelling; the model is whatever the roster names for this leg (the maintainer's example roster saysgemini-3.8-flash-medium), and only when the account catalogue offers it. The-highsuffix IS the effort; do not pass--effort(measured: omitting it works, and a mismatched value is a hard CLI error, so the flag can only break the run or restate the suffix). Verify from the log which model actually served; a model with no calibration rows yet is not a gate merely because it is newer (the runtime file has 3.8's rows). - Model family — the CLI may expose a second family's pool (e.g. Claude)
on separate quota, a legitimate cross-family reviewer for anything that
family did NOT write.
--effortstays absent there too — it is rejected outright for Claude models (see runtime).
Verify the TARGET before launching — git -C "$REVIEW_TARGET_DIR" rev-parse
HEAD must equal $REVIEW_HEAD (a wrong-target launch from a stale cwd is
measured in the codex twin of this skill; here the same mistake is a wrong
--add-dir, which is why the check names the directory explicitly).
Because plan mode is behavioral rather than enforced, bracket the run:
git -C "$REVIEW_TARGET_DIR" rev-parse HEAD # before, and again after —
git -C "$REVIEW_TARGET_DIR" status --porcelain=v1 # -C, or you bracket the
# lead's own cwd, not the targetRunning this leg as a subagent? You are a leaf — block, do not "wait".
agy -p is a foreground CLI, so handing it to the host's background mechanism
and then ending your turn on "waiting for the notification" abandons the run —
nothing will wake you. Poll to exit inside a single tool call, and issue
another immediately if it times out (dev-lead Phase
2).
Verifying the target is not verifying the SPAN. The check above proves
you launched at the right commit; it says nothing about which range the
prompt tells the delegate to read. For a topic branch that range is the
merge-base — never the target branch name — and the two spellings look
identical but are not: git log A..B is "commits in B not in A" (what you
want), but git diff A..B means git diff A B, not merge-base.
Measured: a prompt naming a 2-commit chain but reading it with a two-dot
diff, while the target was 10 commits past the branch point, handed the
delegate two other teams' files as part of the author's change — nothing
errored, both HEAD assertions passed, and the contamination surfaced only
when findings were checked file-by-file against the real diff.
Pin the base once and hand the delegate that, or use three dots:
BASE=$(git merge-base origin/main HEAD)
git diff --stat "$BASE" HEAD # file list must match the change under reviewAny changed HEAD or unexpected dirty-state delta is a failed safety
check — preserve the frozen target and inspect before trusting or relaying a
single line of the review output.
Writing the prompt
The same discipline as any red-team review, and it matters more than the model:
- Tell it to falsify, not to confirm — but frame it as first-party work. Open with "this is a routine pre-merge correctness review of our own code by the team that wrote it", then "for each property below, say whether it HOLDS or is BROKEN, and if broken, exactly how." Avoid "RED TEAM" and "attack" as the opening register — see the refusal gotcha below.
- List the claims explicitly. Quote the properties the code says it has, and ask for each by name. Vague "review this" gets vague output; a numbered claim list gets numbered rebuttals.
- Name the attack surface you already worry about — crash points, partial writes, integer overflow, concurrent paths. It will find more, but not if the prompt reads like a code-style request.
- Demand a trigger and a
file:lineper finding, and ask it to say in one line which claims survived. - State the review-only intent in the prompt as well as the flags, and forbid recursive CLI invocation.
- Forbid running tests/builds, and give the impulse somewhere to go: "if confirming a finding requires execution, report it as a finding that names the exact command and the result that would confirm it — we run it." Note the runtime's honest caveat: if your allow-list contains a test runner (globally — there is no per-role scoping), a disobedient reviewer can actually execute it, while every other build command dies silently with zero output. Both outcomes are wrong; the prompt line stays mandatory.
- A brief may only name commands on this machine's allow-list, and a
pipeline is only as permitted as its least-permitted stage. The
runtime file's one-time
permissions.allowsetup is a prerequisite: with it the delegate runsgit show/log/diff/status/branch/rev-parse; without it even read-onlygit logdies with zero output. What bites after that setup is the pipeline —git show … | sed …dies onsed, silently, on a machine wheregit showis allowed. When the material needs a tool that is not on the list, materialize it yourself rather than widening the list for one run:references/materializing-evidence.mdcarries the four properties that step must have and a worked example. - Ask what the tests do not enumerate, in those words (see
docs/methodology.md— measured as the highest-yield sentence in the prompt across every family). - Forbid praise explicitly. Without it the default register drifts to summarizing and complimenting.
- Size the evidence gate to what YOU cannot check — and on a DOCUMENT (ADR, spec, contract) that is a different list than on code. Demanding re-derivation of facts the author already verified turns a document read into a repo sweep that blows the print timeout with zero output (measured). Keep the unguessable anchor; ask instead what is stated-but-unverifiable, what a reader stopping mid-document would wrongly conclude, and which claims are the author's own inference.
- This leg's measured strength is ATTACK PATHS — brief it on sequences. In paired rounds against a free-pool leg on the same targets, this leg found the bearer credential an intermediary defeats, the single-slot race, and the rate-limit pre-check that bounded nothing — each with a concrete sequence — while the other leg passed all three. The mirror also held: the other leg caught document-consistency defects this one passed. Phrase properties for this leg as "an attacker does X, walk it" and route "does this contradict document Y" to a consistency-strong leg. (Your own calibration journal may differ — measure.)
- Give every claim an evidence gate with an unguessable anchor. This
family is where the motivating incident happened: a format-compliant
all-HOLDS verdict in three minutes, having cited nothing. Require per file
its line count AND the verbatim last line; per claim a
file:lineplus the quoted code; state thatNOT REACHEDis acceptable and HOLDS-without-quote is not.
Reporting back
- Relay findings faithfully — do not soften or drop them.
- Verify every finding against the code yourself before acting. It is confidently wrong sometimes: in the run that motivated this skill, six of eight findings were real (two severe and genuinely subtle), one was wrong on its own premise, one was real but astronomically unreachable. Say plainly which you confirmed, which you reject, and why — and hold a rejection to the same evidence standard the finding was held to.
- Grep every quote before you use it. The frame tells the leg to quote
only text it opened, but measured the same day it was adopted: a framed leg
cited lines 131-189 of a 92-line file and quoted requirement text that
exists nowhere, on a documentation packet, while its conclusions were
partly right; on a code packet its quotes were exact but it put a test in
the wrong change and called a present build line missing. A quote that is
not in the file at the cited lines voids that finding; an attribution is
checked against
COMMITS.txtandDIFF.patch, and an absence claim against the files, before either is relayed. - Review-only: do not apply fixes in the same breath unless asked.
The refusal gotcha
Red-team framing can trip a refusal. "You are a RED TEAM reviewer… falsify claims" over safety-critical code (control systems, medical) has come back as a refusal — the model reads it as an attack on a third party. Reframe as what it actually is: a pre-merge correctness review of first-party code by its own team, with HOLDS/BROKEN verdicts per property. The findings are the same; only the framing changes.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

