agentleFS
on-keydayon-keyday/agent-harness/.claude/skills/dummy-harness/SKILL.md

dummy-harness

Skill2 starsChanged 2 days agoMITView on GitHub

Use when a change needs checking against a running harness rather than a unit test — interactive wake, TUI/WebUI behaviour across a reconnect, "does the operator actually see this". Covers scripts/dummy-harness.sh, the environment traps that make a dummy instance fail in misleading ways, and the two-instance topology needed to test a client across a server restart. Repo-local: this skill is about THIS repository's binaries and scripts.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 0 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Always drive it through eval "$(scripts/dummy-harness.sh env)" rather than reading the state file yourself: the env output starts with …

    line 58
  2. Very specific:

    Every task shell has one. harness-cli prefers a ticket over the operator secret, so pointing it at a dummy server fails with psk auth: psk: …

    line 65
  3. Very specific:

    Every instance gets a bash profile alongside the agent one.

    line 171

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Standing up a throwaway harness to verify against

This skill is not embedded into other repositories' agents — it is about this repository's own binaries (github.com/on-keyday/agent-harness). For driving a live session once you have one (snapshot / send / exec), see session-debugging, which is embedded and applies anywhere.

Why bother

Some properties only exist live and cannot be unit-tested honestly:

  • an agent's terminal UI actually submitting an injected keystroke as a turn (a cooperating fake proves the plumbing, never the agent);
  • what a client does when its server restarts;
  • whether an operator can actually see a thing on the TUI or WebUI.

The alternative — checking against the live fleet — is worse than useless for verifying a change: its server runs a different build on a different host, so a green result there says nothing about your code.

The script

scripts/dummy-harness.sh up [--agent claude|fake] [--model NAME] [--detach] [--name N] [--tls] [--known-psk]
scripts/dummy-harness.sh env  [--name N]     # eval this
scripts/dummy-harness.sh down [--name N]

--tls serves wss:// and an https:// WebUI with a throwaway self-signed certificate for 127.0.0.1 (needs openssl); the instance's CID is then wss:. Chromium's certificate interstitial takes thisisunsafe typed on the page.

For a browser check, start it with --known-psk. The instance then uses the operator PSK written in the script (dummy-not-a-secret), and up prints the WebUI URL (env exports it as WEBUI_URL); hand that URL to Playwright as it is. Without the flag the PSK is random, and building the URL means reading a credential, which an agent's permission check refuses — it cannot tell a dummy's secret from the live one. The known PSK costs nothing real: the instance listens on loopback only and is gone on down.

Loopback only, ephemeral port, fresh operator secret, a fresh runner key enrolled in the temp data dir before the server starts; down removes all of it. The WebUI is served from webui/ on disk, not from the server binary's embedded copy — that default is inverted here on purpose, because an embedded WebUI is frozen at compile time and a JS/wasm edit would then show the OLD page until make build plus a fresh instance. A stale page reads as "my change did not work", which is the worst way to lose an afternoon. --agent claude runs a real claude (default model is the cheapest one) — use it when the property under test involves a real agent's behaviour. --agent fake emits claude's stream-json shapes on a timer, for checks that must not depend on a model or the network.

Always drive it through eval "$(scripts/dummy-harness.sh env)" rather than reading the state file yourself: the env output starts with unset lines that matter (below).

Four traps, in the order you will hit them

1. Your shell's inherited HARNESS_AUTH_TICKET is for the LIVE server. Every task shell has one. harness-cli prefers a ticket over the operator secret, so pointing it at a dummy server fails with psk auth: psk: server rejected: BadTicket — which reads like a secret mistake and sends you to check the secret. The env subcommand emits the unsets for this reason; if you construct the environment by hand, scrub HARNESS_* yourself.

2. CLAUDE_CODE_* / CLAUDECODE / AI_AGENT are inherited too. A claude spawned with those set decides it is a child session and silently disables transcript saving, so --continue / --resume later find nothing. The script scrubs them; the symptom if you don't is a banner reading ⚠ Transcript saving is off — inherited CLAUDE_CODE_CHILD_SESSION marker.

3. agent-runner's --max-tasks defaults to 1. An interactive session holds its slot for its entire life, so the moment you open one, every follow-up foreground open is refused runner_busy and every --background one sits Queued forever. The second symptom is silence — no error anywhere, the task simply never runs. ls shows tasks=1/1 and Busy; check that before debugging anything else. The script passes --max-tasks 4.

4. A restart's effect on in-flight tasks depends on HOW you stopped the server. A deliberate stop — SIGTERM, or the --shutdown-file sentinel, as scripts/restart.py and the dummy's own down use — holds them: the runner keeps the children alive, the tasks read held between the two servers, and the next server re-adopts them (a session comes back detached). A crash — kill -9 — recovers nothing and leaves the old behaviour: Failed with err="runner_disconnected", or err="server_restart" from the replay's own sweep.

So output stopping after a restart is no longer self-explanatory. Check ls first: held means the mechanism is working and you are inside the window; Failed err="hold_expired" means the window passed before a server came back; Failed err="not_held_by_runner" means the runner reported nothing for it. --hold-window=0 on the server restores the old kill-everything behaviour if that is what a test needs.

Two traps specific to testing the hold itself: the window is 90s by default, so a debugging pause between the stop and the restart will expire it and look like a bug; and a held session's output resumes only after re-adoption, so a screen that stops at the restart is expected until the runner reconnects.

Testing a client across a server restart

The window you are watching through must not live on the server you are about to bounce. Two instances:

scripts/dummy-harness.sh up --detach --name host   --agent fake
scripts/dummy-harness.sh up --detach --name target --agent fake

Open an interactive bash session on host (every instance ships a bash profile), start the client inside it pointed at target, then restart target's server. The host session — and your snapshot/send access to it — is untouched.

Inside that host session the shell still carries host's own HARNESS_*, so unset them there before running a client against target:

session exec "$HSID" "unset HARNESS_AUTH_TICKET HARNESS_TASK_ID HARNESS_SERVER_CID; export HARNESS_OPERATOR_PSK='<target operator psk>'; echo ready"

Restarting target's server means killing its pid and relaunching harness-server with the same --listen port, --operator-psk and --data-dir — the data dir holds the enrolled runner key — so the runner and the client reconnect to what they think is the same server.

Driving harness-tui specifically

General TUI technique is in session-debugging — in particular, read the reverse span out of snapshot --style instead of guessing where focus is. Harness-specific facts:

  • A detached session's PTY is 0 0 until a client attaches. The size only ever arrives as a TerminalWindowSize frame from an attached client, which reads its own terminal (objtrsf/exec/exec.go, winsize_unix.go); nobody sends one to a session you are only driving with send/snapshot. So harness-tui started that way dies instantly with terminal too small (need at least 80x24).
  • Size it with --resize, NOT with stty. This section said stty until 2026-08-24; it predates session resize and is wrong for anything you then intend to READ, because the server never learns a size set that way. The mechanism is in session-debugging under session resize — the harness- specific part is only that harness-tui is the program this bites hardest, since it refuses to start at all below 80x24:

    harness-cli session send --resize 45x200 --snapshot --settle-ms 3000 "$ID" ''
  • Once the TUI attaches to a session of its own, that nested PTY inherits the TUI's size, so stty size inside the attached shell is a free check that the handover really propagated the winsize.
  • Tab order is <shown table>, notify, cmdresult, cmdline: the runners, tasks and schedules tables are ONE stop, switched with ← / → (\x1b[D / \x1b[C) while a table has focus. Tab is +1, Shift-Tab (\x1b[Z) is -1.
  • Initial focus is tasks. A reflexive Tab on startup moves focus off the panel you want.
  • Enter on the tasks panel does nothing (it used to follow the task log); d opens the selected task's detail popup. A task's output is read with session snapshot / the grid view while it lives, transcript get or record get afterwards.

The bash profile is your agent-side hands

Every instance gets a bash profile alongside the agent one. A one-shot session opened on it runs its prompt as a shell command (bash -c <prompt>) with that task's HARNESS_* set, which is the only way to exercise agent-side commands (harness-cli agent send, agent inbox, …) without minting a ticket by hand:

harness-cli --server-cid "$CID" session new -d --oneshot --repo "$REPO" --agent bash \
  --prompt "harness-cli agent send --topic chat.<first8-of-target-task> --data hello"

The session ends when the command does, with its exit code. Its output is gone once it ends unless you opened it with --record (record get <id>); read session snapshot <id> while it runs, or make the command's result visible some other way (an agent send, a file in the worktree). A long --prompt "sleep 3600" keeps a task running with a worktree, for testing forwards or file transfer against it.

That is also how you deliver an agentboard message to a session you are watching, since the wake fires on delivery.

The bin behind that profile is resolved, not the bare name, and it is shared with the real bash runner preset (agent_presets.bash_bin()) so a dummy cannot pass a check the thing it stands in for fails. It matters on Windows: bare bash there is C:/Windows/System32/bash.exe, the WSL launcher, and a task dispatched through it dies with execvpe(/bin/bash) failed inside a Linux namespace holding none of the checkout. Git for Windows' bash is used instead. On a machine where none is found, up says so on stderr and ships no bash profile — so --agent bash is then rejected by name at open rather than failing per task.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.