Prompt Injection Detector

Purpose

Detect prompt injection, jailbreak, and adversarial input patterns in user inputs, system prompts, and retrieved context before they reach the LLM pipeline.

Input: Sample inputs (user messages, RAG chunks, tool outputs), application flow description, trust boundaries
Output: Injection Detection Report with pattern catalog, risk ratings, test cases, and mitigation recommendations
Examples: See examples.md for worked input/output.

Workflow

Step 1: Map input surfaces

Identify all text entry points:

Include: chat input, file uploads, RAG retrieval, tool/API responses, email/web scraping, multi-turn history.

Step 2: Classify attack vectors

Scan inputs against known categories:

Step 3: Build pattern catalog

For each detected or anticipated pattern document:

Severity scale: Critical (system override), High (data leak/tool abuse), Medium (behavior drift), Low (noise).

Step 4: Create adversarial test suite

Design test cases per surface:

Include benign false-positive checks (legitimate use of words like "ignore" in domain context).

Step 5: Assess pipeline gaps

Review current defenses:

• Input filtering (regex, ML classifier)
• Prompt structure (role separation, delimiters)
• Output validation
• Tool permission boundaries

Step 6: Recommend mitigations

Prioritize controls by severity × likelihood. Link each gap to a concrete control.

Step 7: Validate

Run Validation checklist.

Decision Rules

Validation

• All input surfaces enumerated with trust levels
• Pattern catalog covers direct, indirect, and encoding evasion
• Each Critical/High pattern has at least one test case
• False-positive scenarios documented for domain-specific language
• Mitigations mapped to specific pipeline stages (pre-LLM, post-LLM, tool gate)
• Severity ratings use consistent scale
• No implementation code (framework-specific)
• Gaps without mitigations flagged as open risks

Anti-patterns

• Regex-only defense — trivially bypassed by paraphrasing and encoding.
• Testing happy path only — missing indirect injection via RAG and tool outputs.
• Ignoring multi-turn context — earlier benign messages used to smuggle instructions later.
• Treating detection as prevention — reporting patterns without pipeline placement for controls.
• Zero false-positive planning — blocking legitimate domain terms without allowlist strategy.

Best Practices

• Apply OWASP LLM01 (Prompt Injection) taxonomy as baseline.
• Separate trusted system instructions from untrusted user/RAG content with explicit delimiters.
• Test with automated red-team payloads and human-crafted edge cases.
• Log detection events with pattern ID for tuning; never log full secrets or PII.
• Layer defenses: input filter → structured prompt → output validator → tool permission gate.
• Re-run detection suite after prompt or RAG schema changes.

Output Structure

# Injection Detection Report: [Application Name]

## Input Surfaces
| Surface | Trust | Current controls |
|---------|-------|------------------|

## Pattern Catalog
| ID | Category | Severity | Detection method | Example |
|----|----------|----------|------------------|---------|

## Test Suite
| Test ID | Surface | Payload | Expected | Status |
|---------|---------|---------|----------|--------|

## Gap Analysis
| Gap | Severity | Affected surface | Recommended control |
|-----|----------|------------------|---------------------|

## False-Positive Notes
- [Domain terms that may trigger filters]

## Open Risks
- [ ] [Unmitigated item]

Next Skills