AI Red Team

When to Use

• Red-teaming chatbots, agents, RAG systems, or copilots before launch
• Designing safety evaluation suites and adversarial test harnesses
• Reproducing reported prompt injection or jailbreak vulnerabilities
• Validating mitigations after incidents (retesting filters, hardening)
• Running multi-turn coercion, encoding, or indirect injection campaigns
• Assessing bias, harmful output, or data exfiltration risks in LLM applications
• Scoping rules of engagement and severity rubrics for AI security testing

When NOT to Use

• Writing corporate AI policy or risk governance frameworks → ai-risk-governance
• Building production LLM features or RAG pipelines → ai-engineer
• General network/AD/infra penetration testing → network-pentester
• Authorized web/API OWASP testing (non-LLM) → web-pentester
• Enterprise adversary simulation, MITRE ATT&CK campaigns, purple team → red-team-specialist
• Binary, firmware, or protocol reverse engineering → reverse-engineer
• CI/CD pipeline security → devsecops

Related skills

Core Workflows

1. Scope and rules of engagement

1. Define target: model, app surface, tools, data stores
2. Obtain written authorization and time window
3. Agree out-of-scope (e.g., no social engineering of employees unless approved)
4. Define success criteria: critical findings, reproduction steps, severity rubric
5. Plan safe test environment (no prod customer data)

See references/engagement_scope.md for ROE template and severity definitions.

2. Threat model for LLM applications

See references/attack_catalog.md for technique families and test prompts (use ethically).

3. Test execution

Phases:

1. Baseline — document intended refusals and allowed behaviors
2. Automated sweep — harness with curated attack set + fuzz mutations
3. Manual creativity — domain-specific abuse scenarios
4. Tool/RAG focus — indirect injection via retrieved documents
5. Regression — re-run after mitigations

Log: input, output, tool calls, latency, whether guardrail fired.

See references/testing_harness.md for harness design and datasets.

4. Reporting

Each finding includes:

• Title and severity (impact × likelihood)
• Steps to reproduce (minimal)
• Evidence (redacted transcripts)
• Affected component
• Recommended mitigation
• Retest criteria

See references/reporting.md for report template and remediation tracking.

5. Mitigation validation

See references/mitigations.md for defense depth and known weak controls.

When to load references

• ROE and scope → references/engagement_scope.md
• Attack types → references/attack_catalog.md
• Harness and automation → references/testing_harness.md
• Reports → references/reporting.md
• Defenses → references/mitigations.md