llm-app-architecture
Skill0 starsChanged 5 months agoMITView on GitHub
Architect a production LLM application — prompt and context design, tool use / function calling, agentic loops, guardrails, evaluation, latency and cost control, and failure handling. Use when building an LLM-powered feature, agent, assistant, or workflow. Complements rag-system-design (retrieval) and ml-system-design (training/serving).
In this repository12 other files it gives its agents.
- adr-writerSkill
- architecture-reviewSkill
- gcp-architectureSkill
- incident-postmortemSkill
- interview-modeSkill
- leadership-lensSkill
- learning-coachSkill
- mlops-pipeline-reviewSkill
- ml-system-designSkill
- observability-reviewSkill
- principal-code-reviewSkill
- rag-system-designSkill
The file
Files
LLM Application Architecture
Design for non-determinism: assume the model can be wrong, slow, or unavailable, and engineer around it.
Prompt & context
- System / developer / user role separation
- Context-window budgeting; what to include vs retrieve
- Structured output (JSON schema / tool calls) over free text
Capabilities
- Tool use / function calling — clear schemas, validated args
- Agentic loops — termination conditions, step caps, loop detection
- Memory / state management across turns
Guardrails
- Input validation & prompt-injection defense
- Output validation (schema, policy, PII)
- Human-in-the-loop for high-impact actions
Evaluation
- Offline eval set with graded rubrics (LLM-as-judge + assertions)
- Regression tests in CI on prompt/model changes
- Online: track refusals, tool-call failures, user corrections
Latency & cost
- Streaming for perceived latency
- Model tiering (small for routing, large for hard tasks)
- Caching (prompt prefix / semantic) and token budgets
Failure handling
- Timeouts, retries with backoff, fallbacks
- Graceful degradation when the provider is down
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.