imprint-teach-deepdive
Skill24 starsChanged 33 days agoMITView on GitHub
Analyze where an `imprint teach` run spent its time — macro phase breakdown from the Phoenix trace plus turn-by-turn per-tool compile timing — and identify the fixable time sinks.
In this repository17 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- commitSkill
- debug-teachSkill
- imprint-example-refresh-releaseSkill
- imprint-google-flights-live-auditSkill
- imprint-reteach-auditSkill
- pr-review-convergenceSkill
- prSkill
- releaseSkill
- commitSkill
- debug-teachSkill
- imprint-auth-compile-logSkill
- imprint-reteach-auditSkill
- imprint-teach-deepdiveSkill
- prSkill
- releaseSkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 19
Use this to answer "where did the ~N-minute teach go, turn by turn, and what's fixable."
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
imprint teach deep-dive
Use this to answer "where did the ~N-minute teach go, turn by turn, and what's fixable."
Two data sources
- Macro (phase/span timing + cost) — Phoenix. Requires
IMPRINT_TRACE=1+ a running Phoenix collector (the teach must have been run with tracing on;scripts/teach-from-scratch.shsets this). Project nameimprint, root spancli.teach. - Micro (turn-by-turn per tool) — the per-tool compile log at
~/.imprint/<site>/<tool>/.compile-log.json(a Codex-cli stream-json event array:{type:'assistant'|'user'|..., message.content[], timestamp}).
Procedure
# 1. Macro: list recent teach traces, pick the site/run, get the span tree
bun run scripts/analyze-phoenix.ts --kind teach --last 8
# Span tree under cli.teach: teach.detect_tool_candidates, teach.plan_prereqs
# (build-plan), build_shared_module[_shared/*], plan_tool[*],
# compile.generate[*] (the dominant phase), compile.playbook[*].
# Cost is on llm.cost.* attrs; sum is the trace total.
# 2. Micro: turn-by-turn for one tool (or sweep the whole site)
bun run scripts/analyze-compile-log.ts ~/.imprint/<site>/<tool>/.compile-log.json --full
bun run scripts/analyze-compile-log.ts --site <site> # all toolsanalyze-compile-log.ts prints: tool-call breakdown, exploration %, done attempts (and how many were rejected), live-integration/pacing time, top time sinks, and the full per-turn timeline with wall durations (delta between consecutive tool_result timestamps).
What the numbers usually show (and the general levers)
compile.generateof heavy RPC tools dominates (e.g. Google batchexecute search/booking ≈ 20–25m each). Most of that is per-tool field-index discovery on a schema-less positional array — the tool walks the decoded payload to find where its fields live. This is inherent, NOT a missing helper: the generic decode/round-trip already exists as a_shared/module and is imported (verify withgrep -n "_shared" ~/.imprint/<site>/<tool>/parser.ts). Do not "recommend adding a decode helper" — that double-counts the shared-modules phase.- 25s live-request pacing on every integration/chain run, serialized inside each compile (anti-flag throttle to the real site). Large, but only partly controllable — a shared token-bucket across lanes can overlap the waits without raising the aggregate request rate.
donerejections each cost a full verify cycle. Two recurring causes: (a) missingparam:<name>coverage tests, (b)noUncheckedIndexedAccessTS errors that the agent's ownrun_testsnever surfaces (it runs unit tests, nottsc). General fixes: runtscinsiderun_tests; add a pre-doneper-param lint.- Concurrency is capped at 2. The critical path = sequential prefix (triage + build-plan + the shared-module build, gated by the slowest module) + the single slowest tool chain. More lanes can't beat that floor.
- A
/tmp/dbg.tsscratch script can'timport '../_shared/...'(relative path doesn't resolve from /tmp) → a wasted iteration. Scratch scripts belong in the tool dir.
Gotchas
analyze-compile-log.tshandles both the current stream-json shape (type+message.content) and the legacyrole/contentshape. Tool names are MCP-prefixed (mcp__imprint-compile__run_bash→run_bash).- Per-turn durations come from the
user-entrytimestamps; assistant entries have no timestamp, so a turn's duration is the time from the previous result to this one (think + tool exec). - Phoenix
compute sum > wallis normal — shared modules build in parallel and plan/playbook overlap adjacent compiles under concurrency-2.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

