transcript-audit

The operator has months of transcripts. Somewhere in them is an accurate account of how they work: what they ask for and how, where they lose hours, which instincts are good, which habits are quietly expensive, and which corrections they have now typed forty times without ever converting into a rule that would have prevented them.

Nobody has read it. They cannot — nobody rereads their own year. Neither can you; the corpus is gigabytes and your window is not. So the question this skill answers is not "what is in the transcripts". It is: what does someone learn about themselves that no single session could ever show them, and what changes because of it.

Three things follow from that, and they are the whole design.

A single session is invisible to itself. From inside one, everything felt productive. The findings worth having are all of the form this happened the ninth time, in the fifth project, over four months — and that sentence can only be written from outside. So almost everything here is counting, and the counting is done for you.

A count is a hypothesis someone had. Every regex in the layer below is a guess by whoever wrote it, which means the instrument can confirm and deny guesses and cannot surface what nobody thought to look for. Closing that hole is your job, not the tool's, and there is a command built for exactly it (stats --what vocabulary).

And the output is not a document about someone. It is a set of changes to the machine they work in, each earned by evidence they can check, each approvable in one click. A report of twelve observations and two actions is a blog post with their name on it.

The one rule, and it is not the one you think

The rule is you choose what enters your context; the corpus never chooses.

A regex you wrote is you choosing. A count you asked for is you choosing. A window you opened because a number pointed at it is you choosing. cat-ing a transcript to see what's in there is the corpus choosing, and one such read pollutes every judgement you make afterwards with material you did not select and cannot unsee.

This is a sharper rule than the one it replaces, and the difference was measured. An earlier version of this file said raw transcript text may never enter your window, full stop. That version was run against a real corpus alongside an unbounded reader with the same mission and no tooling. The unbounded reader produced four findings the bounded one could not reach, and every one of them came from the same place: the agent's half of the conversation. Not from browsing, not from vibes — from being able to read what the agent said right before the operator got angry. The absolute ban was not protecting the report. It was hiding half the evidence.

So there are two doors into original text, both of which return counts or a window and neither of which will ever hand you a corpus to scroll:

• grep '<pattern>' --lane agent — one regex over both lanes of every transcript, returning hits, sessions, projects, months, and a handful of verbatim examples. This is the only way to measure anything in what the agent said.
• zoom --sid X --grep '<quote>' — a handful of turns of original text, uncut, at a place a number already pointed you to.

They chain: grep prints a session id beside every verbatim hit and zoom opens it. That chain was silently broken for one of the two supported clients until recently — the printed id was a flat twelve characters, every Codex transcript begins rollout-2026, so zoom opened an arbitrary different session and reported that the quote did not appear there, offering the clip as the explanation. The id is now long enough to resolve and an ambiguous one is refused out loud. If you ever see a tool here explain your result rather than return it, distrust the explanation first.

Use them when a count has raised a question. Do not use them to go and see what's around. If you cannot say which number sent you, you are browsing.

Both halves of the conversation are evidence

Most of the layer below is built around what the operator typed, because an audit of how someone works is mostly about them. That leaves a blind spot with a shape, and it is worth knowing before you plan: the operator's most expensive recurring defect usually lives in the agent lane, and the operator lane only records him complaining about it.

On the corpus this was built against, one phrase family in the agent lane — a component built and never connected to anything — turned up in 190 agent turns across 144 sessions and 16 projects. Nothing in any slice could see it. The operator's own words on the subject were a handful of exasperated sentences that read like an ordinary bad afternoon. The whole shape of his most durable problem was on the other side of the table.

grep --lane agent is how you look there. Aim it at what agents claim ("done", "verified", "all tests pass"), at what they hedge ("one honest caveat", "I did not actually"), and at the vocabulary of the failure you already suspect.

The layer that does the reading

tools/transcript_audit.py (stdlib only, no install). --help gives the surface.

Slices: prompts corrections interrupts commands tool_errors dispatches invocations slash rework sessions praise. Each is JSONL, each a different question about the same history. Seats may write throwaway Python straight over them.

Five flags that decide whether a number means anything

These are on the rows because each one, missing, has produced a published finding that was false. They are cheap to respect and they are not optional.

• kind — whether a record is a session someone sat in (operator) or the transcript of an agent that was dispatched (subagent). This is the largest denominator hazard in the tool and it went unprinted for three versions. On one corpus 1,117 of 2,364 records were subagents, 832 of the 1,328 that did work, and a dispatched agent cannot be redirected or interrupted because nobody is there to do it — so every one is a structural zero in the numerator and a full unit in the denominator. The published redirect rate read 0.148; the operator's own sessions ran 0.395. stats --what shape now splits it, read and diff take --kind, and diff defaults to operator.
• dup — the same turn recorded twice. /resume forks a transcript and copies everything typed before the fork into a new session id, and nothing in either file says so. A fifth of typed turns can be duplicates. Two independent readers each built a finding on the inflation before catching it; one reported a correction class spanning "12 sessions and 5 projects" that was ten sentences. Aggregates already exclude them, and so does read --exclude dup,brief — without which a reader once counted 157 rows against a printed table's 137 and had no way to see the two disagree.
• brief — an operator turn that is a dispatch packet for another agent, not an instruction to this one. It matters because a packet is written in the grammar of a rebuke — "you did NOT build it", "do not trust any claim" — so the redirect scorer files the operator's best writing as his worst moment. Two seats independently found a third of corrections --sample top was verifier briefs. Excluded from the steering table; still greppable, and often the clearest statement of intent in the corpus.
• pasted — operator-lane text he did not write: another model's output, relayed by clipboard. No harness marker catches it because he really did put it there. It can be a fifth of the characters in his lane. An unbounded reader built a headline about his epistemic standards out of vocabulary that was his agents'.
• truncated / said_len / type_len / cmd_len — long fields are stored clipped with their true length beside them. corrections[].after.said holds 240 characters of an agent turn whose median is 2,243. Never compute a rate over the presence of something in a clipped field. Use zoom for the text.

Two more limits with no flag to warn you. A third of "sessions" can be records that did no work — pipeline calls, health checks, aborted resumes — arriving in bursts on single days and sitting in every per-session denominator; stats --what shape counts them. And no slice can see a Read tool call's contents, so compliance with any "read this file first" instruction is structurally unmeasurable. Say so rather than inferring it.

And one that is not a property of the corpus but of you: every run of this tool becomes transcripts in the corpus the next run searches, written in the corpus's own analytical vocabulary. It is not hypothetical. On one run a seat proved a phrase had zero corpus hits; the same grep twenty minutes later returned four, three of them that audit's own sessions including the re-check itself, and the falsifier found that the draft's highest-impact finding rested on a quote whose speaker was a previous run of this tool rather than the operator. grep excludes its own lineage by default and prints the count it skipped; --until <date> pins an absence claim to a corpus state that predates the run recording it. An absence proved after the audit started is an absence the audit falsified.

praise is the weakest slice: it keys on approving-sounding openers and mostly catches ordinary instructions. A lead, never evidence of taste. Note what its emptiness itself means — if every specific sentence an operator has written about quality is a rejection, their agents can model what they hate in detail and have nothing to learn what they like from. That is a finding, and it lives in the absence.

The steering classes are a lead, not a map. Each correction carries a steer list out of eight declared at the top of the tool. Around a quarter of corrections match anything, and that share prints above the table on purpose. The number that matters is never the hit count; it is how many separate sessions and projects a class spans. Read the quotes behind a row before building on it, and write "I could not classify most of them" rather than dressing 23% as a theory of how someone thinks.

Work backwards from the report

Before you plan anything, imagine the finished report as its reader meets it: the one document that finally read everything, telling this specific person what is going right in a way that proves it looked, what is going wrong with receipts attached, what keeps recurring that no session could see, and what to change, ranked by what next month gains.

Every corpus supports a different report. Different facets dominate, different tensions surface, and the one finding that makes the whole thing worth it comes from somewhere you did not plan. So do not start from the slices and ask what they can produce — start from the report this corpus deserves and ask what would have to be counted, read or compared to earn each part of it. The slices, the stats and the seats are instruments, not an outline. If the evidence supports an angle no seat was pointed at, that angle belongs in the report, and the seats' notes fields are usually where it first shows up.

The loop

0 — Scan, then look at the shape before you think. scan (add --project X or --limit N if scoped), then ls. Then run every stats --what in sequence and read the numbers yourself. Free, deterministic, and where most real findings are born. Do not dispatch anyone until you can state, in your own words, three things the arithmetic already told you.

Two of these are yours and are not delegated. trajectory is one table you already hold, and a mind seated to find change in a corpus that switched client or schema mid-span will find change that is really a schema — check any movement against the volume column and against shape before believing it. And vocabulary is the one place the instrument has no opinion: read it holding no theory, take a term you cannot explain, and aim a read --grep at it. If the aggregates surprise you, that surprise is the most valuable thing in the run. Chase it before anything else.

If a previous run's remedies were applied, run diff --since <that run> before forming any view. A remedy that measurably worked or measurably didn't is the strongest finding available, and it is free.

1 — Seat the minds. Read references/seats.md and dispatch. Each seat is a different person with a different appetite, pointed at different material, working in parallel. You are not sending them to fetch answers you already have; you are giving them a corner of a life and letting them notice what they notice. The shape of the return is yours. The content is theirs, and a seat that comes back with something you didn't anticipate is the seat working, not misfiring.

2 — Synthesise, and be ruthless. Findings collide, duplicate and contradict. Three seats noticing one thing from three angles is one strong finding, not three weak ones — merge them and let the strongest evidence from each survive. Rank by what would change the operator's next month, not by how clever it was to notice. Cut anything you cannot defend with a receipt. Expect to cut half.

Three things about the seam the returns arrive through, each of which has cost a run:

• Read every notes field before you read a single finding. It is where a seat puts what the schema had no room for. On one full run three of the strongest shipped items came from notes rather than findings, including the one the operator acted on first. On another, the notes fields collectively identified six defects in the instrument, two of which had corrupted findings inside that same run — better work than any single finding, and none of it fits the schema. Ask for it explicitly in every packet.
• A payload can arrive corrupted and still look fine. Returns cross a subagent boundary that may HTML-escape them: && comes back as &&, < as <. A heredoc or a redirect turns into something that will not run. Never paste a seat's payload into findings.json unread. Normalise the entities, and for anything touching code or config, open the target file and re-derive the payload against what is there — which is also how you discover the seat proposed a change to a file that already does it.
• Verify absence claims yourself, here, before they reach the falsifier. "This rule is written nowhere", "there is no wrapper", "the template is undocumented" — each is a claim about a whole filesystem made from a slice. Open the file. On one run the single item that survived both falsification and operator approval was an absence claim that was simply false, and it died in front of the operator at apply time. Before any of them, answer one question: where does this operator actually keep what they know? On a later run every salvage finding died at once because all of them had checked the repo, the project CLAUDE.md files and ~/.claude/*/memory/, and this operator's durable knowledge was 670 provenance-stamped notes in an Obsidian vault that a SessionStart hook injected into every session. One of those notes was the exact decision a seat proposed to rescue, dated, revised since, and marked locked. The harness's memory directory is frequently not the operator's memory. Find the real store from the hooks in settings.json and from what a session arrives holding, write its path into every packet, and treat any "undocumented" claim that has not been tested against it as unverified.

3 — Falsify with a mind that never saw the work. Dispatch the falsifier with your draft findings and nothing about how you got there. Its job is to kill: generic advice in specific clothes, statistics that don't mean what you said, remedies that would break something, findings that are really the operator's deliberate choice. Whatever survives, ships. Whatever dies, dies quietly — do not argue it back in.

4 — Write findings.json, render, hand over. Schema, categories and the specificity bar are in references/findings.md. Render, then give the operator the file path and one short paragraph. Do not narrate the findings in chat; the report is the artifact. Then stop and wait.

5 — The apply turn. They come back with a pasted decisions blob. Run apply --findings f.json --decisions '<paste>' for the work order, then execute it: edit the CLAUDE.md, write the skill, install the hook, build the tool. Operator notes override the payload where they conflict. Items they skipped are dead — do not re-pitch them. Verify, then write the ledger entry.

Two things that only show up on an apply turn. Verify by driving the thing, not by re-reading the diff — render the session that crashed, run every verb of the script you installed, measure the before and after. An edit that applied cleanly is not a change that works. And check where a remedy lands: a target inside ~/.claude/skills/ is a copy that the next install deletes and rewrites. If a change is meant to last, it belongs in the repo the skill was installed from.

The laws live in one file, and it is not this one

references/craft.md holds the doctrine: what a corpus does to the person counting it, how to run the seats, what the two lanes are each good for, what an apply turn owes. Every law in it was earned by a run that got it wrong first, and it is the file to read at step 0 and to edit at close. It is stated once, there, so that nothing here can drift out of agreement with it.

The three you are most likely to need before you have read it: the most expensive missing precondition produces no errors, only ritual; spread beats volume, because a motion in one session is an accident somebody was present for; and a learning that stays in prose gets re-broken in the same direction, which is why so much of what used to be advice in this file is now a default in the tool.

What a remedy is allowed to be

Findings without remedies are complaints. Every actionable item carries the exact artifact:

• memory / CLAUDE.md — the literal lines, and which file (global vs project), written so they'd change behaviour next session. One-line rules with the reason attached beat paragraphs. If a rule can't be stated in two lines, it's a skill, not a memory.
• skill — name, description, what it replaces. Only when a workflow recurs and has enough shape to be worth invoking by name. A skill nobody triggers is worse than none.
• automation — a hook, script, alias, cron or settings entry that removes a motion performed by hand N times. Cite N and its spread. stats --what automation exists for this and it is the highest-yield category in the report.
• harness config — permissions that stop repeated prompts, env vars, model or effort defaults, MCP wiring. Small, checkable, immediately felt.
• build — real code worth writing, scoped to what one session could finish.
• recover — a decision or hard-won constraint settled in a transcript and written nowhere durable. One session-expiry from being re-derived. Say where it should live.
• retire — machinery that exists and is not used: a skill never invoked, a doc never read, a rule the transcripts show being ignored. Deleting is a deliverable.

Two categories carry no remedy and both earn their place: Holding (what they do well, said precisely enough that they know you looked — not flattery, but telling someone which of their instincts to trust) and Do differently (a habit change with no artifact).

The bar, restated because it is the whole product

Before an item goes in the file, read it as the operator: would they say "how did it know that", or "this is a blog post"? If there is any doubt, it is a blog post. Everything ordinary you were about to write has already been written by someone who never saw their corpus. Yours is the only version with the receipts. Spend them.

Compounding

Two files, and the difference between them is kind, not privacy. Read both at step 0.

• references/craft.md is doctrine: laws for mining any corpus, each earned by a run that got it wrong first. It ships with the tool. It is rewritten when a law is superseded, never appended to, and a law that becomes enforceable in code gets deleted from it rather than left as ballast. Propose an edit here only when a lesson would help a stranger auditing somebody else's history.
• ~/.transcript-audit/LEDGER.md is the log: append-only, local, never published, and untouched by an install because it sits beside the runs. Everything that names a project, a client, a path or a person. Which of this operator's habits a previous run already ruled on. Which remedies were applied and whether the next diff said they worked. The corpus-shape numbers a future dispatch packet should not spend budget rediscovering.

Close by appending to the log. Then ask the harder question: did you learn anything about the instrument rather than the operator? That is the most valuable thing you can leave, because it is the half that can become code — and if it can become code, it should, because a lesson that stays in prose gets re-broken. This skill has the receipts on itself for that.

A note on what this produces

A finished report contains verbatim quotes from someone's entire working history, and findings.json contains them too. Both belong wherever that person keeps private things, never in a repo. The tool never sends anything anywhere, and selftest deliberately prints shapes rather than text because its output gets pasted into bug reports.

References

• references/seats.md — the minds, their appetites, the dispatch packet. Read before step 1.
• references/findings.md — the findings.json schema, categories, specificity bar, and a worked example of a finding that passes versus one that looks like it should.
• references/craft.md — the laws for mining a corpus. Skim at step 0, with the local log.