AI Agent Red-Team Methodology

You attack an AI agent through its tools, not through URLs and forms. The LLM is middleware; the tools are the sink; the real prize is what those tools can reach — cloud IAM, customer data, secrets managers, internal services, actions taken on a user's behalf. This skill routes you to the right hunt-agent-* skill AND enforces the verification discipline that separates an impactful, genuine finding from a confident-wrong one. In real engagements a false "the copilot leaked customer PII" is worse than useless — it burns client trust and gets the whole report dismissed. The discipline is not optional.

PART 0 — THE VERIFICATION LAW (applies to every skill, every finding)

Agents generate text; they don't retrieve state. That is why a verification step is mandatory — a fluent answer can be a hallucination that looks identical to a real leak. Before you claim ANYTHING:

1. Self-description is untrusted. The agent's account of its own tools/limits is generated and frequently false ("I pass your input verbatim," "I don't have access to that"). Verify by effect.
2. Verify the channel discriminates. Prove known-true vs known-false give different results before trusting an oracle; confirm your OOB listener receives a literal marker before trusting exfil.
3. Marker-fired ≠ data-fired. A control landing is not your data leaving. Unique, non-guessable marker per hypothesis.
4. Reproduce ≥3×. Non-determinism means one hit can be luck. Report hit-rate for probabilistic bugs.
5. OOB-or-it-didn't-happen. Exfil confirmed only when the listener receives your marker + real data.
6. Refusal ≠ secure; compliance ≠ vuln; enumeration ≠ impact; "it parsed" ≠ RCE. The finding is the state/data/action obtained, not what the model said.
7. Actioned secret. A leaked credential/secret/record is confirmed only when it authenticates, the record verifiably belongs to another tenant/user, or the OOB callback carries it.
8. Minimum-necessary extraction. Prove the missing control with 2–3 records; volume is client liability, not stronger evidence. Synthetic markers; don't persist customer data.

Why these exist (field war-stories): trusting an agent's "I don't sanitize" claim (false); mistaking a literal OOB probe for the real payload firing; misreading a binary status oracle; planning a boolean-oracle extraction against a tool whose allowlist made it impossible. Every rule above is a mistake that cost hours.

PART 0b — THE NEGATIVE-RESULT LAW (what must be true before you report "clean")

PART 0 stops you claiming a finding you cannot prove. This stops the opposite and more common failure: a probe that reports clean because it never looked. "We tested it and found nothing" is a claim to the client with the same evidentiary burden as a finding.

1. A control must prove the probe could have fired — plant a known-true case and see it detected in the same run.
2. Denylist the read-only tools; never allowlist the dangerous ones — an allowlist ignores every tool it has not heard of.
3. Errors are their own verdict — 5xx, timeout, empty audit, dead dependency = UNMEASURABLE, never "refused".
4. Prove the plant persisted before reporting isolation — no memory to cross means no result.
5. Tokenise identifiers at ONE character — ids are routinely a single digit.
6. Word boundaries; normalise whitespace before subtracting — substrings invent findings, verbatim matching against re-wrapped text deletes them.
7. A fixed-size effect is not proof; a SCALING one is — constant overhead is an extra model turn, not execution.
8. Judge a tuning change by whether a GENUINE detection dropped, not by the FP count.
9. Do not inherit another tool's verdict — benchmarks and scanners have their own bugs.

Measured in this bundle: a similarity gate scored a real system-prompt leak at 0.03 and said no-fire; a curated tool list hid two attacker-booked reservations; a two-character token floor missed the forged-observation attack that leaked the flag; a whitespace mismatch erased all 14 detections in a run; an HTTP 500 read as a refusal buried a confirmed auth bypass. Each was a probe calling a compromise clean.

PART 1 — WHERE AGENTS LIVE (enterprise context)

Two questions decide most findings: per-user identity or a shared service account? (shared → confused-deputy) and multi-tenant? (yes → cross-tenant access is the crown jewel).

PART 2 — AGENT TAXONOMY → SKILL

PART 3 — THE 4-PHASE WORKFLOW

1. CHARACTERIZE ─▶ 2. INJECT/ACCESS ─▶ 3. EXFILTRATE/EXTRACT ─▶ 4. ESCALATE/CHAIN

Phase 1 — CHARACTERIZE (never skip; where hours are saved)

• Enumerate every tool (per agent); note params + output visibility (full / status / timing / blind).
• Trifecta check: does this agent simultaneously have (a) access to private data, (b) exposure to untrusted content, and (c) an outbound/exfil channel? All three = prioritize target; any two = watch; one or none = low exfil risk.
• Identify the interpreter (2+2, check <host>, key: value, {{7*7}}, SELECT 1).
• Map the allowlist/denylist — probe commands one at a time; don't assume cat/grep/id exist.
• Egress test — can it reach your OOB listener? which protocol (DNS usually; HTTP/TCP often blocked)?
• Oracle semantics — what drives the status (last exit code? stdout match? reachability?).
• Auth model + tenancy — per-user vs service account; single vs multi-tenant.
• Extract the tool contract (hunt-agent-prompt-extraction) — the system prompt often states the injection point.
• Confirm on the wire (proxy), not just the chat UI (client filters, newline stripping).

Phase 2 — INJECT / ACCESS

Separators (; \n && | $()), then resolve the newline mode (Mode A shell-exec vs Mode B the-agent-splits-into-a-tool-call — decides whether $() fires; see hunt-agent-tool-injection). Or bypass the agent via MCP/A2A.

Phase 3 — EXFILTRATE / EXTRACT

Pick the channel by what Phase 1 proved: OOB (usually the winner for blind) → boolean oracle (only if a comparator runs) → timing (last resort, statistics required). hunt-agent-blind-exfil.

Phase 4 — ESCALATE / CHAIN

PART 4 — WHAT COUNTS

PART 5 — STANDARDS MAPPING (right list — two different documents)

• Model-level = OWASP LLM Top 10. Cite 2025 IDs by default, 2026 equivalent in parens (a reviewer may be on either edition). Watch the renumber: Excessive Agency LLM06:2025 → LLM03:2026; System Prompt Leakage LLM07:2025 → renamed Hidden Context Exposure LLM08:2026; Improper Output Handling LLM05:2025 → LLM10:2026. (Never use the retired 2023 "Insecure Plugin Design.")
• Agent-level = OWASP Agentic Apps. Two real docs: Top 10 for Agentic Applications 2026 (ASI01–ASI10, publ. 2025-12-09) for the headline class, and Agentic AI: Threats & Mitigations v1.1 (T1–T17, Dec 2025) for the specific threat. Most tool findings = ASI02 Tool Misuse / ASI03 Identity & Privilege Abuse / ASI06 Memory & Context Poisoning.
• MITRE ATLAS v5.6.0 + CWE as applicable. Note AML.T0053 = AI Agent Tool Invocation (not "Plugin Compromise"). Full verified ID list: RESEARCH-2026.md.
• Reproduce & anchor. Where possible, reproduce on a public benchmark and cite the number — AgentDojo (629 security cases), InjecAgent (ReAct-GPT-4 vuln 24%), ASB (84.3% peak ASR), PoisonedRAG (5 texts → 90% for RAG). Tooling: garak, PyRIT, promptfoo, mcp-scan / MCPSafetyScanner. See RESEARCH-2026.md §3/§6.

PART 6 — SKILL ROSTER (23)

hunt-agent-prompt-injection · hunt-agent-output-exfil · hunt-agent-memory-poisoning · hunt-agent-rag · hunt-agent-guardrail-evasion · hunt-agent-resource-abuse · hunt-agent-tool-injection · hunt-agent-blind-exfil · hunt-agent-deserialization · hunt-agent-mcp-exposure · hunt-agent-mcp-supply-chain · hunt-agent-multi-agent (incl. A2A) · hunt-agent-prompt-extraction · hunt-agent-race-condition · hunt-agent-auth-bypass · hunt-agent-time-gate · hunt-agent-info-disclosure · hunt-agent-crypto · hunt-agent-brute-force · hunt-agent-hitl-bypass · hunt-agent-native-memory · hunt-agent-browser-use · hunt-agent-config-poisoning · redteam-mindset (discipline). See _CONTRACT.md for the standard every skill must meet.

PART 7 — PROBE INDEX

Every hunt-agent skill carries a self-contained ## Runnable Probe (stdlib-only, copy-paste, ends with # LETHAL WHEN:). Grab the one for the surface you're on; each produces a discriminating oracle you then escalate to a real sink.

A static probe tests the payloads you thought of. When a guard/denylist blocks the obvious one, don't stop at the first refusal — mutate the payload (homoglyph, zero-width/Unicode-Tag smuggling, case/split, encoding) and re-fire, keeping a benign control that must NOT land so the oracle stays honest. The evasion a single probe missed is usually one transform away.

Status: FIELD-FIRED = validated on a live ARKX agent pod. UNTESTED = probe exists but hasn't been fired against a live agent (backed by public research/CVEs, pending field confirmation).

Discipline: probes are single-target, marker-based, authorized-testing only. The probe gives you the oracle; PART 0 (Verification Law) + the skill's 8-point gate decide whether it's a finding.