LLM and Agent Security — Trust Boundary for Models That Can Act

Use this skill when a change lets a model see untrusted content, or lets an agent component act with real credentials. Source checks and dates: SOURCES.md. Incident facts and links: Incident Appendix.

Contents

• Two places this skill applies
• Architecture
• Quick Reference
• The #1 Mistake: The Model Authorizes Itself
• Decision Matrix: "Where Does the Control Live?"
• Decision Matrix: "Is This Secure?"
• OWASP LLM Top 10 (2026)
• OWASP Agentic Top 10 (2026)
• Review Checklist
• What This Skill Does Not Contain

Where a cheat sheet still describes a delimiter, a filter, or a second model call as the defense, follow the 2026 Top 10. Those controls are partial. OWASP LLM01:2026 states that prompt injection is intrinsic: a model does not separate instructions from data, and no reliable prevention exists today (consistent with NIST 2025, NCSC 2025, and Debenedetti et al., 2025). Design the system so a bypassed instruction boundary cannot move credentials, data, or side effects.

The 2026 LLM list renumbered several 2025 entries. Excessive Agency is LLM03:2026 (it was LLM06:2025). Supply Chain is LLM04:2026. Improper Output Handling is LLM10:2026. The Agentic Top 10 PDF still cites the 2025 numbers in places. This skill uses the 2026 ids.

HTTP authentication, SSRF allowlists, CSP, and the Next.js AI SDK tool handler stay in api-security. The signed approval on that handler is AI Tools. This skill is the rest of the boundary: what enters the prompt, memory, compaction, install-time packages, MCP identity, and output sinks.

Two places this skill applies

Instruction files (CLAUDE.md, AGENTS.md, .cursor/rules, .cursorrules, .aider.conf.yml) are on the builder surface. A cloned repo that an agent treats as policy is a trust decision.

Architecture

Untrusted content (user, page, ticket, tool result, memory, peer agent)
        │
        ▼
Model call with the minimum tools for that call
        │
        ▼
Policy broker in application code  ← authorization, schema, tenant, budget
        │
        ▼
Typed side effect (user-scoped credential, single purpose)
        │
        ▼
Human approval row, when the effect is irreversible or leaves the system

The model is an untrusted principal. The broker does not read "the user approved this" out of the transcript, a summary, or another agent's message.

Quick Reference

The #1 Mistake: The Model Authorizes Itself

A system prompt, a "do not follow instructions in the document" line, or a filter on the user message does not hold once the model has a tool and a credential.

// FAIL — the model picks the user, the SQL, and whether to send
const result = await model.complete({
  system: 'You are a support agent. Never refund more than the order total.',
  input: ticket.body,
  tools: [{ name: 'runSql', parameters: { query: 'string' } }, { name: 'sendEmail' }],
})

// PASS — the model may only propose a typed action. The broker checks ownership.
const proposal = await model.complete({
  system: 'Choose an order id from the list. Do not invent one.',
  input: JSON.stringify({ question, orders: ownedOrders }),
  tools: [{ name: 'proposeRefund', parameters: RefundArgsSchema }],
})
await proposeRefund(proposal.toolArgs)

proposeRefund is implemented in Human Approval. It loads the session, checks the order, and stores an approval row. The model's sentence is not the approval.

Decision Matrix: "Where Does the Control Live?"

Decision Matrix: "Is This Secure?"

OWASP LLM Top 10 (2026)

OWASP Agentic Top 10 (2026)

Review Checklist

• Every block in the prompt is labeled with its source, and untrusted blocks are data
• The call that sees untrusted content does not also hold a send, pay, delete, deploy, or secret-writing tool
• Tools are an allowlist of typed functions with a schema checked in application code
• No shell, eval, raw SQL, or open URL tool is offered to the model
• Credentials on the tool are the end user's, short-lived, and single-purpose
• Ownership and tenant checks run in the broker, using the session, not a model argument
• Irreversible or outbound actions require an approval row whose text the model did not write
• Memory and schedules are written by application code from structured user actions
• MCP servers are pinned. Tool allowlists are rechecked after any transform. OAuth issuer is pinned
• Agent state is keyed by the authenticated tenant, not by a caller-supplied string
• AI dependencies are hash-pinned. Lifecycle scripts do not run on a host that holds production secrets
• Repo-local agent config in untrusted trees is not executed
• Model output is encoded for its sink. Remote images and link previews are off unless allowlisted
• Secrets, tokens, and connection strings are absent from the system prompt, tools list, and logs
• Retrieval authorizes each chunk before it reaches the model
• A budget caps model spend per user and per job
• The change was compared with the incident appendix

What This Skill Does Not Contain

Reproduction of injections, jailbreaks, worms, or malware. Each incident entry states the impact and the containment control, then links the vendor or researcher report. Incident-response command sequences stay in those advisories.

Sources

Canonical list and check date: SOURCES.md.