agentleFS
ashaychangwaniashaychangwani/imprint/.agents/skills/imprint-teach-deepdive/SKILL.md

imprint-teach-deepdive

Skill24 starsChanged 33 days agoMITView on GitHub

Analyze where an `imprint teach` run spent its time — macro phase breakdown from the Phoenix trace plus turn-by-turn per-tool compile timing — and identify the fixable time sinks.

The file

We split this skill into the rules it gives an agent

1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this to answer "where did the ~N-minute teach go, turn by turn, and what's fixable."

    line 19

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

imprint teach deep-dive

Use this to answer "where did the ~N-minute teach go, turn by turn, and what's fixable."

Two data sources

  1. Macro (phase/span timing + cost) — Phoenix. Requires IMPRINT_TRACE=1 + a running Phoenix collector (the teach must have been run with tracing on; scripts/teach-from-scratch.sh sets this). Project name imprint, root span cli.teach.
  2. Micro (turn-by-turn per tool) — the per-tool compile log at ~/.imprint/<site>/<tool>/.compile-log.json (a Codex-cli stream-json event array: {type:'assistant'|'user'|..., message.content[], timestamp}).

Procedure

# 1. Macro: list recent teach traces, pick the site/run, get the span tree
bun run scripts/analyze-phoenix.ts --kind teach --last 8
#    Span tree under cli.teach: teach.detect_tool_candidates, teach.plan_prereqs
#    (build-plan), build_shared_module[_shared/*], plan_tool[*],
#    compile.generate[*] (the dominant phase), compile.playbook[*].
#    Cost is on llm.cost.* attrs; sum is the trace total.

# 2. Micro: turn-by-turn for one tool (or sweep the whole site)
bun run scripts/analyze-compile-log.ts ~/.imprint/<site>/<tool>/.compile-log.json --full
bun run scripts/analyze-compile-log.ts --site <site>          # all tools

analyze-compile-log.ts prints: tool-call breakdown, exploration %, done attempts (and how many were rejected), live-integration/pacing time, top time sinks, and the full per-turn timeline with wall durations (delta between consecutive tool_result timestamps).

What the numbers usually show (and the general levers)

  • compile.generate of heavy RPC tools dominates (e.g. Google batchexecute search/booking ≈ 20–25m each). Most of that is per-tool field-index discovery on a schema-less positional array — the tool walks the decoded payload to find where its fields live. This is inherent, NOT a missing helper: the generic decode/round-trip already exists as a _shared/ module and is imported (verify with grep -n "_shared" ~/.imprint/<site>/<tool>/parser.ts). Do not "recommend adding a decode helper" — that double-counts the shared-modules phase.
  • 25s live-request pacing on every integration/chain run, serialized inside each compile (anti-flag throttle to the real site). Large, but only partly controllable — a shared token-bucket across lanes can overlap the waits without raising the aggregate request rate.
  • done rejections each cost a full verify cycle. Two recurring causes: (a) missing param:<name> coverage tests, (b) noUncheckedIndexedAccess TS errors that the agent's own run_tests never surfaces (it runs unit tests, not tsc). General fixes: run tsc inside run_tests; add a pre-done per-param lint.
  • Concurrency is capped at 2. The critical path = sequential prefix (triage + build-plan + the shared-module build, gated by the slowest module) + the single slowest tool chain. More lanes can't beat that floor.
  • A /tmp/dbg.ts scratch script can't import '../_shared/...' (relative path doesn't resolve from /tmp) → a wasted iteration. Scratch scripts belong in the tool dir.

Gotchas

  • analyze-compile-log.ts handles both the current stream-json shape (type + message.content) and the legacy role/content shape. Tool names are MCP-prefixed (mcp__imprint-compile__run_bash → run_bash).
  • Per-turn durations come from the user-entry timestamps; assistant entries have no timestamp, so a turn's duration is the time from the previous result to this one (think + tool exec).
  • Phoenix compute sum > wall is normal — shared modules build in parallel and plan/playbook overlap adjacent compiles under concurrency-2.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.