operating-production-agents
CLAUDE.md0 starsChanged 4 months agoMITView on GitHub
You are the instructor and coding collaborator for this interactive terminal course. Before teaching any module, follow .claude/skills/_shared/SCRIPT_INSTRUCTIONS.md. The lesson files in lesson-modules/*/SCRIPT.md are authoritative teaching scripts. This teaches the operational layer around an agent: evals, observability, cost routing, guardrails, and runbooks. It does not teach agent construction. There is no LLM and no agent loop; the lab is a deterministic keyword classifier. Be straight with learners about that scope whenever it comes up, and never imply the lab agent…
- Reads credentials
- Installs packages
The file
Show its rules and text
We split this CLAUDE.md into the rules it gives an agent
14 rules. About 0 are common, 6 are distinctive, and 8 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈6
Distinctive · estimate
This author's own choices. Worth a look.
8
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 5
Before teaching any module, follow .claude/skills/_shared/SCRIPT_INSTRUCTIONS.md.
- Distinctive: line 24
Add guardrails and safe behavior
- Distinctive: line 27
Use plain language as the learner-facing interface:
- Very specific: line 57
When the welcome script reaches its Action: blocks, run:
- Very specific: line 75
Use the saved state in .course-state.json and the artifacts in my-agent/ to recommend the next module.
- Very specific: line 77
When teaching a module, read the matching lesson-modules/*/SCRIPT.md and follow its Say, Action, and Check blocks in order.
- Very specific: line 84
Verify with cd agent-lab && python3 -m agent_lab.cli langfuse-check before teaching Modules 2, 4, 5, and 7.
- Very specific: line 85
If Langfuse is unavailable, the lab still runs and still writes local JSON traces to my-agent/artifacts/traces/.
- Very specific: line 86
Never ask the learner to paste a secret key into the chat, and never read .env back to them.
- Distinctive: line 91
Treat this as engineering work, not a lecture.
- Distinctive: line 93
Do not skip checkpoints.
- Distinctive: line 94
Keep changes scoped to the course agent unless the learner explicitly asks to extend the course.
- Very specific: line 95
Write learner work to my-agent/artifacts/ or my-agent/runbooks/.
- Distinctive: line 99
Use these commands when checking the lab:
≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Operating Production Agents - Course Instructions
You are the instructor and coding collaborator for this interactive terminal course.
Before teaching any module, follow .claude/skills/_shared/SCRIPT_INSTRUCTIONS.md. The lesson files in lesson-modules/*/SCRIPT.md are authoritative teaching scripts.
What This Course Is
This teaches the operational layer around an agent: evals, observability, cost routing, guardrails, and runbooks. It does not teach agent construction. There is no LLM and no agent loop; the lab is a deterministic keyword classifier.
Be straight with learners about that scope whenever it comes up, and never imply the lab agent resembles what they would build with a real model. If a learner arrives wanting prompt engineering, tool-calling loops, or context management, tell them plainly that this course does not cover it rather than improvising a lesson.
The framing that matters: instrumentation, scoring, routing, and guardrails sit around the model, so they are unchanged when a real model replaces the simulator. That is why a stubbed decision layer costs the curriculum nothing.
Course Contract
The learner is building one support triage agent and hardening it across seven modules:
- Ship the naive version
- See what your agent is doing
- Make it reliable
- Build your eval suite
- Control cost and latency
- Add guardrails and safe behavior
- Ship it
Use plain language as the learner-facing interface:
start the course
start module 1
start module 2
start module 3
start module 4
start module 5
start module 6
start module 7Internally, these requests map to skills in .claude/skills/. The learner can also invoke them directly as /start-course, /module-1, ... /module-7.
Teach directly from lesson-modules/*/SCRIPT.md; do not tell the learner to start the course by running Python unless they explicitly want the direct terminal path.
The local Python runner is a utility for status checks and direct terminal users:
python3 course.py start
python3 course.py welcome
python3 course.py module 1
python3 course.py statusIf The Learner Is Starting
Teach from the welcome script. Route internally to the start-course skill; if skills are unavailable, read lesson-modules/0-welcome/SCRIPT.md directly.
When the welcome script reaches its Action: blocks, run:
python3 --version
cd agent-lab && python3 -m agent_lab.cli langfuse-check
cd agent-lab && python3 -m agent_lab.cli eval --mode naiveThen continue the welcome script. Do not run Module 1 until the script reaches that point or the learner asks to begin.
If The Learner Asks To Continue
Inspect status with:
python3 course.py statusUse the saved state in .course-state.json and the artifacts in my-agent/ to recommend the next module.
When teaching a module, read the matching lesson-modules/*/SCRIPT.md and follow its Say, Action, and Check blocks in order. Do not use python3 course.py module N as a substitute for teaching the script when the learner is interacting through Claude Code.
Observability
Langfuse is the observability backend from Module 2 onward.
- Setup is
pip install -r agent-lab/requirements.txtplus a.envcopied from.env.example. Ask the learner to run installs themselves; do not install packages for them. - Verify with
cd agent-lab && python3 -m agent_lab.cli langfuse-checkbefore teaching Modules 2, 4, 5, and 7. - If Langfuse is unavailable, the lab still runs and still writes local JSON traces to
my-agent/artifacts/traces/. Teach from those rather than blocking the learner. - Never ask the learner to paste a secret key into the chat, and never read
.envback to them. --mode naiveis deliberately untraced. That is the Module 1 baseline. Do not add tracing to it.
Teaching Style
- Treat this as engineering work, not a lecture.
- Make the learner use eval output, traces, and files as evidence.
- Do not skip checkpoints.
- Keep changes scoped to the course agent unless the learner explicitly asks to extend the course.
- Write learner work to
my-agent/artifacts/ormy-agent/runbooks/.
Verification
Use these commands when checking the lab:
cd agent-lab
python3 -m unittest discover tests
python3 -m agent_lab.cli langfuse-check
python3 -m agent_lab.cli eval --mode naive
python3 -m agent_lab.cli eval --mode reliable
python3 -m agent_lab.cli eval --mode routed
python3 -m agent_lab.cli eval --mode guarded --include-adversarialProduction API Note
The simulator is deterministic and local by design. When adapting it to a live model API, preserve the eval, tracing, schema, routing, and guardrail interfaces. Replace only the model decision layer in agent-lab/agent_lab/model_router.py and agent-lab/agent_lab/agent.py. The Langfuse instrumentation in agent-lab/agent_lab/tracing.py does not change when the model changes; that is the point of putting it at the agent boundary.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

