Agent Engineering
Skill0 starsChanged 4 months agoMITView on GitHub
Build tool-using LLM agent systems — loop architecture, tool design, memory, and safety. Auto-load when building, extending, or reviewing an agent, ReAct loop, tool-calling system, or autonomous workflow; when designing agent tools or tool registries; when adding agent memory (short-term window, long-term retrieval); when deciding sandboxing/isolation for an agent that can execute code or shell commands; or when asked "should this be an agent?". NOT for parsing/retry mechanics of individual LLM calls (llm-integration-reliability) and NOT for the whether-an-LLM-at-all decision (llm-system-design).
- Deletes or force-pushes
In this repository25 other files it gives its agents.
- API and Interface DesignSkill
- Architecture AnalysisSkill
- Build and ReleaseSkill
- Campaign PlanningSkill
- Change ControlSkill
- Codebase OnboardingSkill
- Code Review PlaybookSkill
- Configuration ManagementSkill
- Debugging PlaybookSkill
- Dependency ManagementSkill
- Documentation PracticesSkill
- Engineering MinimalismSkill
- Evaluation FrameworksSkill
- Failure ArchaeologySkill
- LLM Integration ReliabilitySkill
- LLM System DesignSkill
- Observability and DiagnosticsSkill
- Performance EngineeringSkill
- Prompt and Context EngineeringSkill
- Proof and AnalysisSkill
- Refactoring PlaybookSkill
- Research MethodologySkill
- Security Review PlaybookSkill
- System DesignSkill
- Validation And TestingSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 55
Prefer a self-registering registry (tools register on import/definition) so adding a tool means adding one file, not editing central …
- Distinctive: line 178
Every tool's arguments are validated in code.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Agent Engineering
Purpose
An agent is a loop in which a model chooses actions — which means every defect in the model's judgment becomes a runtime behavior. This skill is the engineering discipline that makes that survivable: hard bounds designed in from day one, small typed tools, memory only when needed, safety proportional to a written threat model, and runs that can be reconstructed after the fact.
Metadata
- Prerequisites:
llm-system-design(confirms an agent loop is actually warranted);security-review-playbook(the threat model that sizes agent safety). - Related Skills:
llm-integration-reliability(per-call reliability inside the loop),prompt-and-context-engineering(the agent's system prompt and context),evaluation-frameworks(measuring agent task success),observability-and-diagnostics(run reconstructability),engineering-minimalism(most "agents" should be pipelines). - Owns: agent loop architecture (bounds, termination); agent tool design and registries; agent memory design; agent sandboxing/isolation decisions.
When to Use / When NOT to Use
Use when:
- Building or modifying an agent loop, its tools, or its memory.
- Reviewing agent code for safety, boundedness, or debuggability.
- Deciding how much isolation an agent needs (guardrails vs container/VM).
- An agent misbehaves at the loop level: runs forever, thrashes between tools, forgets context, or does something destructive.
Do NOT use (load the sibling instead):
- Deciding whether an LLM (or an agent at all) belongs in the system →
llm-system-design. - A single call's output is malformed / needs retries / needs type-guards →
llm-integration-reliability. - Writing the agent's prompts or managing its context window →
prompt-and-context-engineering. - Measuring whether the agent completes tasks correctly →
evaluation-frameworks. - Auditing the security of the host system generally →
security-review-playbook.
Definitions & Mental Model
- Agent loop: repeated cycle of (model proposes action) → (system executes tool) → (observation returned to model), until the model answers or a bound trips.
- Tool: a single-purpose function the model may invoke, described by name, description, and a typed argument schema.
- Observation: the string/structure returned to the model after a tool runs — including tool errors, which are observations, not crashes.
- Bound: any hard limit the model cannot exceed regardless of its choices — iteration caps, per-tool timeouts, token/cost budgets.
- Guardrail vs boundary (definitions owned by
engineering-minimalism; classification procedure bysecurity-review-playbook): a guardrail catches accidents and is bypassable; a boundary cannot be bypassed. Blocklists are guardrails. Containers are boundaries.
Mental model: treat the agent as an enthusiastic intern with root access and no fear. You do not make the intern safe by asking nicely (prompts are guardrails); you make the environment safe by deciding in advance what the intern can touch (tools), how long they may try (bounds), what they can remember (memory), and what room they are locked in (isolation). Every design question below reduces to one of those four. And before all of it: most tasks assigned to "agents" have a knowable step sequence — those are pipelines, and pipelines are easier to test, bound, and price.
Core Methodology
- Confirm the loop is earned. From
llm-system-design: an agent loop is justified only when the step sequence is genuinely unknowable in advance. If you can write the steps down, write a pipeline instead. Record this decision; it is the most common design error in the field. - Design the bounds before the loop. Non-negotiable, in code, on day one:
- Iteration cap (single-digit-to-low-tens; pick a number and enforce it). On trip: return the best partial result plus an explicit "iteration limit reached" marker — never spin.
- Per-tool timeouts, sized per tool (a shell command deserves more than a math eval; a web fetch sits between).
- Budget cap where calls cost money: max tokens or max spend per run.
- Termination condition the model controls (a final-answer action) plus the caps the model cannot control.
- Design tools as small, single-purpose, typed functions.
- One tool, one capability. "run_shell" and "read_file" — never "do_stuff".
- Each tool declares: name, one-sentence description written for the model, and a typed args schema. The description is an interface contract — write it like API documentation (
api-and-interface-designowns interface craft). - Validate tool arguments in code before executing; the model's respect for the schema is a guardrail, your validation is the boundary.
- Prefer a self-registering registry (tools register on import/definition) so adding a tool means adding one file, not editing central dispatch.
- Return tool failures as observations —
"Error running {action}: {e}"— so the loop survives and the model can adapt. (Reliability mechanics:llm-integration-reliabilitystep 9.)
- Add memory only when the task demands it (decision tree):
Does the task span more than one loop run?
├─ NO → context window IS the memory. Add nothing. (Most agents end here.)
└─ YES → What must persist?
├─ Recent conversational state → SHORT-TERM: rolling window of the
│ last N exchanges. A list. No database.
└─ Facts retrievable by meaning across sessions → LONG-TERM:
embed + store + top-k similarity retrieval.
├─ Treat stored blobs as UNTRUSTED at load time
│ (deserialization safety: security-review-playbook)
├─ Fix the embedding model — the vector space is a stable
│ contract; changing it invalidates the store
└─ Design the clear/expiry story before the write path- Size safety to a written threat model. Answer in writing (procedure owned by
security-review-playbook):- What can this agent execute or modify? (shell? filesystem? network? payments?)
- Who chooses the model and the prompts — a trusted operator, or anyone?
- Then choose isolation:
Trusted operator + trusted model + local machine
→ guardrails suffice (blocklists, confirmations, path allowlists)
BUT label them guardrails, with the bypass documented.
Untrusted input reaches the agent (webhooks, user prompts, retrieved web content)
→ prompt injection is now in scope: the model may be turned against you.
Real boundaries required: container/VM, least-privilege credentials,
no destructive tools, egress restrictions.
Agent output triggers irreversible external effects (publish, pay, delete)
→ human-in-the-loop gate or dry-run-first on that tool, regardless of trust.- Make every run reconstructable. Log each cycle: iteration number, proposed action, arguments, observation (truncated but present), and termination reason. The acceptance test (owned by
observability-and-diagnostics): a failed run must be diagnosable from its log alone, without re-running. - Test the loop's failure surface, not its happy path. Fake the model (monkeypatch/stub) and assert: malformed action → strike behavior; unknown tool → survivable observation; tool exception → loop continues; iteration cap → partial answer with marker; blocklist/validation → rejection. (Test mechanics:
validation-and-testing; per-call parsing behavior:llm-integration-reliability.) - Measure task success before shipping changes. Loop tweaks (new tool, new prompt, new bound) are evaluated against a task suite, not vibes —
evaluation-frameworks.
Agent design checklist
- "Should this be a pipeline?" answered in writing — the loop is earned
- Iteration cap, per-tool timeouts, and budget cap in code from day one
- Cap-trip behavior returns partial result + explicit marker
- Every tool single-purpose with typed, code-validated arguments
- Tool errors returned as observations; loop survives any single tool failure
- Memory tier chosen by the decision tree; none is the default
- Long-term store treats its own blobs as untrusted at load
- Written threat model; isolation level matches it; guardrails labeled with bypasses
- Irreversible-effect tools gated (human approval or dry-run-first)
- Every run reconstructable from logs alone
- Failure-surface tests exist (malformed action, unknown tool, tool crash, cap trip)
Discovery & Audit Commands
Audit an existing agent codebase:
# Find the loop and its bounds (absent bounds are the finding)
grep -rn -E "while|for .* in range|max_iter|MAX_ITER|iteration" --include="*.py" --include="*.ts" . | grep -v node_modules | head -20
# Tool inventory: registrations, schemas, dispatch
grep -rn -E "register|TOOL|tools\s*=|args_schema|@tool" --include="*.py" --include="*.ts" . | grep -v node_modules | head -30
# Timeouts per tool (each execution path needs one)
grep -rn -E "timeout" --include="*.py" --include="*.ts" . | grep -v node_modules | head -20
# Destructive capability surface (what can the agent actually do?)
grep -rn -E "subprocess|os\.system|shell=True|exec\(|child_process|unlink|rmtree|requests\.|httpx|fetch\(" --include="*.py" --include="*.ts" . | grep -v node_modules | head -30
# Memory: persistence paths and deserialization safety
grep -rn -E "sqlite|\.db|pickle|np\.load|allow_pickle|embed" --include="*.py" . | head -20
# Is the run reconstructable? (logging of actions/observations)
grep -rn -E "log|console|print" --include="*.py" --include="*.ts" . | grep -iE "action|observation|tool" | head -20Audit rule: every tool execution path must show (a) argument validation, (b) a timeout, (c) error-to-observation conversion. Any path missing one is a defect.
Failure Modes & Anti-patterns
| Symptom | Mistake | Correction |
| Agent runs forever on a confusing task | No iteration cap | Hard cap + partial-answer return with explicit marker (step 2) |
| One tool exception kills a 30-step run | Tool errors propagate as crashes | Catch → return as observation; loop survives (step 3) |
| Agent deleted/overwrote something irreplaceable | Destructive tool with guardrail-only protection | Boundary (isolation/gating) for irreversible effects; guardrails are for accidents (step 5) |
| "It's safe, the prompt says to be careful" | Prompt instructions treated as a security boundary | Prompts are guardrails; isolation and code validation are boundaries |
| Retrieved web content hijacks the agent | Prompt injection outside the threat model | Untrusted input in scope → container/least-privilege/no destructive tools (step 5) |
| Fixed 3-step task built as an agent | Loop chosen for a knowable sequence | Pipeline; agents only for unknowable sequences (step 1) |
| "Why did it do that?" unanswerable | Actions/observations not logged | Log every cycle; runs reconstructable from logs alone (step 6) |
God-tool execute(anything) | Tool surface not decomposed | Small single-purpose tools with typed, validated args (step 3) |
| Embedding store breaks after model swap | Embedding model treated as swappable | Fix the embedding model; the vector space is a contract (step 4) |
| Loop hangs on one slow tool | Global timeout only, or none | Per-tool timeouts sized to each tool (step 2) |
| RCE via tampered memory file | Own DB blobs trusted at load | Safe deserialization; treat stored blobs as untrusted (step 4) |
| New tool required editing 4 dispatch sites | Central hardcoded dispatch | Self-registering registry; a tool is one file (step 3) |
Worked Example
Task: an agent that triages incoming bug reports — reads the report, searches the codebase, and files a labeled ticket.
- Earned? The search path depends on what the report reveals — the sequence is genuinely unknowable. Loop earned. (Filing the ticket alone would be a pipeline.)
- Bounds: 8 iterations; timeouts — code search 10s, file read 5s, ticket API 15s; budget 50k tokens/run.
- Tools:
search_code(query: str),read_file(path: str, max_bytes: int),file_ticket(title: str, body: str, label: enum). Args validated in code —read_filerejects paths outside the repo root (path traversal is a boundary, not a guardrail);file_ticket.labelvalidated against the enum. - Memory: none — each triage is one run; the context window suffices.
- Threat model: bug reports are untrusted input that the model reads → prompt injection in scope. Mitigation: no shell tool, no network beyond the ticket API, least-privilege API credential (create-only), agent runs in a container.
- Irreversible effects:
file_ticketis append-only and labeled by the agent — acceptable without human gate; aclose_tickettool would require one. - Reconstructability: every cycle logged as
iter=N action=search_code args={...} → obs[240 chars]; termination reason logged. - Failure tests: stubbed model emits malformed action twice → run ends surfacing raw output;
search_coderaises → next observation is the error string and the loop continues; 9th iteration → partial triage + "iteration limit" marker.
Repository Examples
Repo facts below are point-in-time illustrations (as of 2026-07-04) — examples, never assumptions about your system.
- agentix (
~/agentix) — a ~700-line local ReAct agent framework embodying most of this skill: self-registering tool registry (Tooldataclass: name, description, args_schema, fn; register-on-import — adding a tool is one file plus an import); hard cap of 10 loop iterations; per-tool timeouts (shell 30s, python 15s, web 15s); tool exceptions returned as observation strings so the loop survives; dual-tier memory (short-term rolling window + long-term SQLite embeddings at~/.agentix/memory.dbwith cosine top-k) with the embedding model fixed (nomic-embed-text) and the chat model selectable; written threat model in the README — "the security boundary is you and the model you point it at, not the code" — with the shell blocklist["rm -rf /", "sudo", "mkfs"]honestly documented as a guardrail (bypass named) and container/VM advised for real isolation; failure-surface tests (malformed-JSON strikes, unknown tool, blocklist) using pytest monkeypatch only. - prism (
~/prism) — the contrast case: an LLM system that is correctly NOT an agent. Its review sequence is knowable (fetch diff → chunk → judge chunks → post), so it is a bounded fan-out pipeline — the step-1 decision made right.
Validation Criteria
You applied this skill correctly when:
- The pipeline-vs-agent decision is recorded, and the loop's step sequence is demonstrably unknowable.
- Reading the code reveals iteration cap, per-tool timeouts, and budget caps without searching hard — and a test trips each.
- Every tool's arguments are validated in code; feeding an out-of-schema call in a test produces a rejection observation, not an execution.
- Killing any single tool mid-run leaves the loop alive with the error visible in the transcript.
- The threat model document exists and the isolation level matches it; every guardrail is labeled with its bypass.
- A teammate can reconstruct any failed run — actions, observations, termination reason — from logs alone.
- Memory, if present, matches the decision tree; if absent, that was the deliberate default.
Provenance & Maintenance
- Sources:
~/agentix(primary case study),~/prism(contrast case) — investigated 2026-07-04. Owner context: primary consumers are Sonnet-class agents; minimalism doctrine owner-confirmed 2026-07-04. Skill authored 2026-07-06; methodology is repo-independent. - Assumptions: agentix constants (10 iterations, 30s/15s/15s timeouts, blocklist contents, memory path) are point-in-time. Prompt-injection guidance reflects the 2026 threat landscape — attacks evolve; re-check.
- Re-verification commands:
grep -rn "register\|args_schema" ~/agentix/agentix | head grep -rn "timeout\|BLOCKED" ~/agentix/agentix/tools | head grep -in "boundary" ~/agentix/README.md - Likely to drift: agentix's caps/timeouts/blocklist; agent-framework idioms industry-wide; the prompt-injection mitigation state of the art.
- Maintenance checklist:
- Re-run re-verification commands; re-stamp Repository Examples.
- Re-check the isolation decision tree against current prompt-injection research annually.
- Confirm cross-referenced skills still exist under their directory names.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

