Kaviar — make the pipeline learn

The pipeline's phases are instructions; instructions do not improve by themselves. This closes the loop: evidence → lesson → approved skill change. It exists so a mistake costs tokens once instead of every session, and so the fix is a written rule rather than a hope.

Read shared/references/note-style.md first — everything here is note-form.

Step 1 — Get the evidence (deterministic, no recall)

node 05-kaviar/scripts/session_retro.cjs                       # this session + its subagents
node 05-kaviar/scripts/session_retro.cjs --workspace <dir>     # newest tree in a workspace
node 05-kaviar/scripts/session_retro.cjs --session <id|path>   # one specific session
node 05-kaviar/scripts/session_retro.cjs --out RETRO-<date>.md

It reads $DSH_HOME/sessions/<workspace>/<sessionId>/session.v3.jsonl.zstd and reports what actually happened: failures with their tool and kind, calls retried verbatim, user pushback, tool histogram, files touched, tokens. Do not summarise the conversation from memory instead — that reconstructs a plausible story rather than the record, and it is the more expensive path.

If it reports no failures, say so and stop. A session with nothing to learn is a valid outcome; inventing a lesson to justify the ritual is not.

Step 2 — Judge (this is the only part that needs a model)

The digest is leads, not verdicts. For each candidate, ask:

• Is it generalisable? "The gate flagged email templates" is a fact about one gate; "every denylist gate needs a curated ignore list or people stop reading it" is a lesson. A rule that only fits the session that produced it makes the skill worse.
• Is it already covered? Search the phase SKILL.md and LESSONS.md first. Re-learning a known lesson is a duplicate, not progress.
• Does it have evidence? A lesson without a session id and a concrete symptom is a guess.

At most three lessons per session. More than three means the session needs a phase change, not a longer list — say which phase and why instead.

Step 3 — Record

node 05-kaviar/scripts/lessons.cjs add --phase 2.5 \
  --symptom "<what happened, in the words of the failure>" \
  --fix "<the rule that prevents it>" --evidence <session-id>

Repeating a symptom reinforces the existing entry (hit count) instead of adding a twin — that hit count is the signal for what deserves promotion. Caps: 100 entries / 8 KiB, enforced by check; prune --keep 40 evicts by hit count (not recency) into LESSONS-archive.md.

Step 4 — Test the wording before it goes near a skill

A lesson is a hypothesis about behaviour. Promotion used to rest on hit count plus judgement, which is a guess with a paper trail. Test it instead — this is the cheap form of "no skill without a failing test first", and it is the whole reason microtest.cjs exists.

node 05-kaviar/scripts/microtest.cjs plan  --scenario 05-kaviar/scenarios/<name>.json
#   → prints the workflow script to run: control + proposed wording, 5 reps each, model pinned
node 05-kaviar/scripts/microtest.cjs score --scenario ... --results <a file with the workflow's return value>

Write a scenario next to the lesson, using the failure that actually happened as the task. Then read the verdict, not the vibe:

Read every flagged output by hand before acting — a scorer cannot tell a real match from an echo of the wording. Worked example with real numbers: scenarios/review-shape.evidence.md. A lesson that is not testable this way (a one-off environment fix, a missing tool) is recorded as a lesson and not promoted into a skill.

Step 5 — Promote, with the user, only if it generalises

node 05-kaviar/scripts/lessons.cjs promote K-003                     # prints the proposal only
node 05-kaviar/scripts/lessons.cjs apply K-003 --skill <SKILL.md> --approved

promote writes nothing. apply refuses without --approved — a lesson the model decided to promote by itself is exactly the failure mode this design exists to prevent (a bad skill change that merges silently costs more than it saves). After applying:

node ../install-harness-skill/scripts/install-skill.cjs --check-only <skills-root>
node ../shared/scripts/note_lint.cjs <edited SKILL.md>

Both must pass — frontmatter rules and BOM issues make a skill silently vanish, and the change would look applied while doing nothing. When the pipeline shares a root with other skills, lint the package rather than the root, or the check reports other people's prose as failures:

node ../shared/scripts/note_lint.cjs --tree <skills-root> --exclude <foreign-skill>

Whole-file rewrites of a SKILL.md are out of scope here. One gotcha line, in the right phase, per approved lesson — and a rule about the shape of output goes in as a recipe, not as another "never".

Step 6 — Report

Three lines: what the session cost, what was learned (ids), what changed. Then stop — kaviar is one bounded pass, never a loop, and never a reason to keep working after the user's task is done.

What this must not become

• Not a session summary. The digest is evidence; the output is rules.
• Not an unsupervised editor. Skill edits require approval; lessons appends do not.
• Not a growing memory. Caps + eviction by hit count + archive; an unbounded lessons file is a known hard failure (oversized skill content has broken agent memory prefetch in the wild).
• Not an excuse to re-terse the skills. Compressing instruction prose is a focus trade, not a saving (shared/references/cache-and-cost.md §7) — never a kaviar outcome.

Files

• scripts/session_retro.cjs — deterministic digest from the session log (has --selftest)
• scripts/lessons.cjs — add/list/check/prune/promote/apply over LESSONS.md
• scripts/microtest.cjs — plan / score / selftest for the wording test in Step 4
• scenarios/ — one JSON per tested rule, plus the evidence file that records the result
• ../shared/LESSONS.md — the shared lessons file; phases may keep their own alongside their SKILL.md
• ../shared/references/instruction-forms.md — which form a rule needs, and why