agentleFS
Seriora-ResearchSeriora-Research/seri-agent/.claude/skills/verify-seri/SKILL.md

verify-seri

Skill0 starsChanged 2 months agoApache-2.0View on GitHub

Drive the seri coding-agent TUI the way a user does — under a real PTY — and capture proof; the handful of real non-interactive surfaces (--version, --help, --selftest, serve, exec) only as the exception. Use when a change to apps/cli needs evidence it works in the real app, not just tests.

  • Reads credentials

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 1 is distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Before driving anything, one read-only check from the repo root:

    line 28
  2. Very specific:

    Verify side effects, not just the screen: a write task's file content afterward.

    line 77

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Verify seri

seri is the CLI in apps/cli, and this project uses it through the TUI: a user runs seri in a real terminal, gets the full-screen session, and does everything there — tasks, approvals, /mode, /model, /exit. The non-interactive seri <task> path exists in the product but is not how seri is used here; do not verify a feature through it when the map gives a TUI path. run() dispatches positional verbs serve, exec, doctor, and update, plus the --version/--help/--selftest flags. Those are the only non-interactive surfaces; seri --help is the authority on that list. config, permissions and usage are TUI slash commands (/config, /permissions, /usage), never subcommands — typing seri config list sends config list to the model as a task and bills a real turn. The monorepo also has apps/server, apps/web, apps/lab, apps/portal; those are separate surfaces this skill does not cover.

Every run in this skill uses a throwaway profile (--profile verify-<run-id>, where <run-id> is unique per run, e.g. verify-20260828a). A profile isolates config.json, auth.json, permissions.yaml, sessions/ and checkpoints/ under ~\.seri\<profile>\ — the user's real state lives in ~\.seri\ directly (the default profile) and must never be driven or deleted. A fresh profile starts with no config, so it resolves provider groq, model openai/gpt-oss-120b, and the GROQ_API_KEY that bun auto-loads from the repo-root .env (env beats config.json). That key is real: a driven turn is a real model call.

Launch

There is no server to keep alive; launch means: deps installed once, then each drive is its own short-lived TUI session in its own PTY.

bun install            # once per checkout; also vendors ripgrep via postinstall
  • TUI session (the primary drive; needs a real PTY — node-pty/ConPTY on Windows): node .claude/skills/verify-seri/scripts/drive-tui.mjs <evidence>\tui.txt verify-<run-id> <step...> node, never bun — bun as the node-pty host on this machine kills every ConPTY child instantly (exit -1073741510 after only ?9001h?1049h-style enables); under node the same drive works, including from agent tool-shells. Do not set SERI_DISABLE_MODELS_FETCH=1; drive the same catalog fetch a user gets. From Git Bash also set MSYS_NO_PATHCONV=1, or type=/exit reaches the app as C:/Program Files/Git/exit. Ready when the transcript shows the splash (the seri banner — mark, version, model and directory — and, on an unauthenticated profile, the login picker with footer ↑/↓ move · Enter select · Esc continue).
  • Non-interactive surfaces (the exception — plain processes, piped stdio): bun apps/cli/src/cli.ts --selftest, --version, --help, and the serve/exec/doctor/update verbs, run from the repo root, stdout + $LASTEXITCODE captured directly. Nothing else. Any other first word is a task.

Doctor

Before driving anything, one read-only check from the repo root:

bun apps/cli/src/cli.ts --selftest

Healthy prints exactly one line, selftest ok: ripgrep <version> (CLI loads, workspace links resolve, vendored ripgrep runs). If it errors with Cannot find module '@seri/daemon-client', run bun install — the workspace links are missing, nothing else is wrong. Also confirm Test-Path .env at the repo root; without it a fresh profile has no Groq key and every real turn fails on auth, which looks like a product bug but is not one.

Drive

TUI. scripts/drive-tui.mjs (invocation under Launch) spawns seri under ConPTY and executes steps in order: wait=TEXT (20s deadline), type=TEXT, key=esc|enter|ctrl-c|ctrl-d|shift-tab, sleep=MS. Extra CLI args (--continue, --resume <id>) go between the profile and a :: separator: drive-tui.mjs <out> <profile> --continue :: <step...>. It always writes the full decoded transcript to its first argument, exits 0 only if every wait= matched, and kills what it spawned (matched by the unique profile name on the command line — one more reason the profile must be unique). A smoke drive:

node .claude/skills/verify-seri/scripts/drive-tui.mjs <evidence>\tui.txt verify-<run-id> `
  "wait=Esc continue" sleep=400 key=esc "wait=created.@30000" sleep=600 `
  type=/exit sleep=400 key=enter sleep=800

And a real model turn (proven live: done · 6324 ↑, 34 ↓ on openai/gpt-oss-120b · your key):

node .claude/skills/verify-seri/scripts/drive-tui.mjs <evidence>\tui.txt verify-<run-id> `
  "wait=Esc continue" sleep=400 key=esc "wait=created.@30000" sleep=600 `
  "type=Reply with exactly the word SERI-TUI-PROOF" "wait=exactly the word" key=enter `
  "wait=done ·@90000" key=ctrl-d sleep=800

The verified drive pattern (each piece below was learned from a live failure — don't simplify it away):

  • Splash dismissal, always first: "wait=Esc continue" sleep=400 key=esc "wait=created.@30000" sleep=600. The created. wait is load-bearing, not decoration: dismissing the splash does not mean the session exists, and until it does the app shows a starting session… placeholder that accepts no input. Typing into that window is what made issue #211 look like a hang. A fresh profile's splash is a login picker (Log in / Sign up / Continue without logging in); a bare Enter on it SELECTS "Log in" and starts a real WorkOS device flow (it can open a browser) — Esc is "continue without logging in". The splash owns the whole terminal and the mode label is not painted under it, so created. — not the mode label — is what proves the session mounted.
  • Submit a task: "type=<task>" "wait=<inner phrase of the task>" key=enter — the echo-wait proves the input box had focus. The wait phrase must be a mid-text fragment of a longer task; short strings (and every slash command) get split by cursor redraw in the byte stream and never match.
  • Slash commands: type=/exit sleep=400 key=enter — type, settle, submit; no echo-wait.
  • Turn completion: "wait=done ·@90000" — a real turn needs the longer deadline (@MS suffix). Once only. A wait= scans the whole accumulated byte stream, so a second wait=done · in the same drive matches the first turn's line and returns instantly, and the drive races on to quit while turn two is still running. For a multi-turn drive, settle the later turns with sleep= or wait on a phrase unique to that turn.
  • Screenshot a drive: scripts/frame-from-transcript.mjs <transcript> <out-prefix> replays the transcript through a minimal VT emulator and writes the final screen as .txt, .json and .ansi; scripts/frame-to-png.ps1 -Json <out-prefix>.json -Out <out-prefix>.png paints the grid. Use it for any change whose proof is what the screen looks like. Parsing stops at the alt-screen exit, so the captured frame is the last one the TUI painted — end the drive with a sleep= and no key=ctrl-d when you want the app at rest rather than its quit screen.
  • Outcome: the helper's stderr DRIVE-RESULT code=0 line plus the transcript are the result; the process exit code and any stdout after the helper in the same shell command are unreliable (the ConPTY plumbing swallows them — observed as exit 255 on fully green drives). Run the helper as its own command and read the transcript in a separate one. The post-quit (tokens: …) summary may flush after capture ends — assert done · <n> ↑, <n> ↓ from the turn status instead.

A write tool in approve-each mode raises the ApprovalBox — [y]es / [a]lways (saved for this project) / [N]o — answered by a single type=y / type=a / type=n keypress, no Enter. Stable handles to wait= on, all rendered by the real app: Esc continue (splash up), ┌ (box border — the TUI renders ┌, never ╭), Session + created. (TUI mounted past the splash), ⏸ approve-each mode on / ⏸ read-only mode on / ⏵⏵ bypass permissions on (mode label, cycled by key=shift-tab), [y]es (ApprovalBox up), done · (turn finished). Prefer these over screen positions.

Probes answer what a driven transcript cannot show, and they assert nothing — a probe run has no pass or fail, only a recording to read. scripts/probe-mouse.mjs <out-dir> <profile> <step...> (node, same ConPTY rules as drive-tui.mjs) spawns the TUI and writes raw.bin, decoded.txt and report.json into the out-dir: every private-mode enable, disable, save and restore in arrival order (the TUI no longer writes ?1007 itself; a leftover save/restore would still show up here), any OSC 52 payload decoded from base64, and the child's exit code. It takes drive-tui.mjs's whole step vocabulary plus drag=C1,R1,C2,R2, click=C,R and wheel=up|down,C,R, which inject SGR 1006 mouse bytes — the only way to script a mouse against seri. scripts/probe-clipboard.mjs (bun, no profile, no PTY) round-trips opentui's native host clipboard and puts back whatever was there; confirm its result with the platform's own reader afterwards, since reading through the same library it wrote with proves nothing on its own. Both were written for docs/specs/044-tui-selection-copy/research.md and are the instruments its numbers came from.

Non-interactive surfaces are the exception for what has no TUI path: --version, --help, --selftest, and the serve/exec/doctor/update verbs (always with --profile verify-<run-id> when they touch state). Capture stdout + exit code directly. Check a candidate against seri --help before driving it; anything absent there is a task, not a command.

Evidence

Evidence for run <run-id> goes to .claude/loops/verify-seri/<run-id>\ — gitignored (.claude/loops/ is per-run local state by this repo's own convention), and it survives cleanup.

Proof standards:

  • Exercise the real user path — the TUI under a PTY, never an imported function, a faked runLoop, or the non-interactive prompt path standing in for a TUI feature. The unit tests already cover the fakes.
  • Capture the action and the result: the full helper command line, the complete transcript file, and the exit code (recorded in a .txt next to the transcript).
  • Verify side effects, not just the screen: a write task's file content afterward; a config change re-read by reopening /config; a session's ~\.seri\verify-<run-id>\sessions\ entry; a persisted grant in /permissions.
  • A model's answer is nondeterministic prose — seed a token (SERI-VERIFY-<run-id>) and assert on it appearing, not on wording.
  • Gate proofs answer the real ApprovalBox in approve-each or read-only. --dangerously-skip-permissions is a real product flag but a run using it proves nothing about the gate.

Cleanup

Kill only what you started; the helper already does this for TUI drives (unique-profile match, never image name). Then:

Remove-Item -Recurse -Force ~\.seri\verify-<run-id>   # the throwaway profile — NEVER ~\.seri itself

plus any probe files or scratch dirs the run created. Evidence under .claude/loops/verify-seri/<run-id>\ is not cleanup's to remove. If a drive fails mid-way, run this same cleanup before retrying so broken attempts don't strand ConPTY children or half-written profiles.

Isolation limits

  • Two verification runs coexist if their profiles differ; never reuse a run-id.
  • seri serve binds a loopback daemon per profile — do not drive a daemon this run did not start, and do not start one on the user's default profile.
  • The default profile's config sets OpenRouter; a throwaway profile deliberately does not inherit it. If a proof specifically needs the user's provider setup, say so and get confirmation instead of driving the default profile silently.

Feature map

features/README.md is the index; one file per user-facing feature is the maintained recipe. A proof that drives one convenient entry point is incomplete when the map lists others. Keep the map honest with /pstack:maintain-verification-skill.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.