AI Threat Modeling

Purpose

Produce a threat model for AI/LLM applications using STRIDE and LINDDUN adapted for model inference, RAG, agents, and tool use.

Input: System architecture (containers, data flows), AI pipeline description, asset inventory, trust boundaries
Output: AI Threat Model document with assets, threats, attack trees, risk ratings, and control recommendations Examples: See examples.md for worked input/output.

Workflow

Step 1: Define scope and assets

Document system boundary and AI-specific assets:

Include: system prompts, fine-tuned weights, embeddings, vector stores, conversation logs, API keys, user PII, tool credentials.

Step 2: Draw trust boundaries

Map data flow across boundaries:

• User → application → LLM provider
• Application → RAG retrieval → document store
• Agent → tools/APIs → external systems
• Training/fine-tuning pipeline (if applicable)

Step 3: Apply STRIDE per component

For each container/component, enumerate threats:

Add AI-specific variants: prompt injection, model inversion, training data poisoning, adversarial inputs.

Step 4: Apply LINDDUN for privacy

For flows involving personal data:

Focus on: conversation retention, embedding reversibility, cross-user retrieval leakage.

Step 5: Build attack trees

For top risks, decompose attack paths:

Goal: Exfiltrate system prompt
├── Direct injection in user input
├── Indirect via poisoned RAG document
└── Tool output manipulation

Step 6: Rate and prioritize

Score each threat: Likelihood (1–5) × Impact (1–5) = Risk score.

Step 7: Recommend controls

Map threats to mitigations: preventive, detective, corrective. Reference OWASP LLM Top 10 where applicable.

Step 8: Validate

Run Validation checklist.

Decision Rules

Validation

• All AI-specific assets listed with sensitivity classification
• Trust boundaries cover LLM provider, RAG, tools, and storage
• STRIDE applied to every in-scope component
• LINDDUN applied where PII or conversation data flows
• Top 5 risks have attack trees or attack path descriptions
• Each High/Critical risk has recommended control
• Controls traceable to OWASP LLM categories where relevant
• Residual risks documented explicitly

Anti-patterns

• Generic STRIDE without AI context — missing prompt injection, RAG poisoning, tool abuse.
• Threats without attack paths — abstract labels without exploitable scenarios.
• Ignoring supply chain — overlooking model provider, plugin, and embedding API risks.
• Single-boundary model — treating the LLM as a black box with no internal data flows.
• Controls without verification — recommending "input validation" without test criteria.

Best Practices

• Combine STRIDE (security) and LINDDUN (privacy) in one unified model.
• Use data-flow diagrams aligned with C4 container level.
• Prioritize threats by exploitability in the actual deployment (internet-facing vs internal).
• Link each control to a measurable verification method (test, monitor, audit).
• Re-run threat model when adding tools, RAG sources, or model changes.
• Involve domain owners for LINDDUN unawareness and compliance gaps.

Output Structure

# AI Threat Model: [System Name]

## Scope
- **In scope:** [components]
- **Out of scope:** [components]

## Assets
| Asset | Sensitivity | Location |
|-------|-------------|----------|

## Trust Boundaries
[Diagram or boundary table]

## STRIDE Analysis
| Component | Threats | Risk | Controls |
|-----------|---------|------|----------|

## LINDDUN Analysis
| Data flow | Privacy threats | Risk | Controls |
|-----------|-----------------|------|----------|

## Attack Trees
### [Top threat name]
[Tree or path description]

## Risk Register
| ID | Threat | Score | Status | Owner |
|----|--------|-------|--------|-------|

## Open Items
- [ ] [Unmitigated threat]

Next Skills