Trace analysis

Reading runs/*.jsonl traces produced by core.trace.Run — the playbook's observability principle applied to debugging this specific repo. "You need visibility into prompt chains, model decision paths, retrieval contexts, token consumption, and the entire reasoning workflow" — this skill is how you actually get that visibility after a run completes.

Workflow

1. Run scripts/summarize_trace.py <path> first rather than reading the raw JSONL by eye — it gives outcome, total cost, per-step cost breakdown, and loop detection in one pass.
2. Check the outcome field before anything else. ok, error, or budget_exceeded changes what you're looking for — an error outcome means read the error field and find the last event before it; budget_exceeded means look at which step's cost climbed fastest.
3. Find cost hotspots by step, not by call count. A step called once at BALANCED tier can cost more than twenty FAST-tier calls — the per-step cost breakdown (not the step count) tells you where to optimize.
4. Treat any step repeated more than ~5 times as a suspected loop, especially in patterns/agent_loop or agents/* traces — check the input_preview/output_preview fields across those repeats to see if the model is actually making progress or stuck retrying the same thing.
5. For multi-agent traces, reconstruct the interaction order from timestamps, not from event order alone — concurrent specialists' events interleave in the file; sort by ts and look for overlapping windows to confirm parallelism actually happened.
6. Compare against a baseline run when diagnosing "why did this cost more." A single trace in isolation tells you what happened; two traces of the same task on different patterns (see evals/) tell you whether the cost is expected for this pattern or an anomaly.

Composition

Stand-alone diagnostic skill. Its output (cost/loop/failure findings) is exactly the kind of structured input business-intelligence can turn into a "here's what to fix" recommendation if the audience needs that framing.

Quality bar

• The outcome field and any error message are checked and reported first.
• Cost is broken down by step, not just given as a single total.
• Any step repeated more than 5 times is explicitly flagged as a possible loop, with evidence (not just a raw count).
• Claims about concurrency are checked against actual timestamps, not assumed from the pattern's design intent.