dummy-harness
Skill2 starsChanged 2 days agoMITView on GitHub
Use when a change needs checking against a running harness rather than a unit test — interactive wake, TUI/WebUI behaviour across a reconnect, "does the operator actually see this". Covers scripts/dummy-harness.sh, the environment traps that make a dummy instance fail in misleading ways, and the two-instance topology needed to test a client across a server restart. Repo-local: this skill is about THIS repository's binaries and scripts.
In this repository22 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- codebase-tourSkill
- harness-cli-from-a-toolSkill
- harness-cliSkill
- independent-reviewSkill
- landing-to-mainSkill
- memory-maintenanceSkill
- running-parallel-workersSkill
- session-debuggingSkill
- supervising-workersSkill
- codebase-tourSkill
- harness-cli-from-a-toolSkill
- harness-cliSkill
- implementation-pitfallsSkill
- independent-reviewSkill
- landing-to-mainSkill
- memory-maintenanceSkill
- running-parallel-workersSkill
- session-debuggingSkill
- supervising-workersSkill
- surface-parity-checklistSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 0 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 58
Always drive it through eval "$(scripts/dummy-harness.sh env)" rather than reading the state file yourself: the env output starts with …
- Very specific: line 65
Every task shell has one. harness-cli prefers a ticket over the operator secret, so pointing it at a dummy server fails with psk auth: psk: …
- Very specific: line 171
Every instance gets a bash profile alongside the agent one.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Standing up a throwaway harness to verify against
This skill is not embedded into other repositories' agents — it is about
this repository's own binaries (github.com/on-keyday/agent-harness). For
driving a live session once you have one (snapshot / send / exec), see
session-debugging, which is embedded and applies anywhere.
Why bother
Some properties only exist live and cannot be unit-tested honestly:
- an agent's terminal UI actually submitting an injected keystroke as a turn (a cooperating fake proves the plumbing, never the agent);
- what a client does when its server restarts;
- whether an operator can actually see a thing on the TUI or WebUI.
The alternative — checking against the live fleet — is worse than useless for verifying a change: its server runs a different build on a different host, so a green result there says nothing about your code.
The script
scripts/dummy-harness.sh up [--agent claude|fake] [--model NAME] [--detach] [--name N] [--tls] [--known-psk]
scripts/dummy-harness.sh env [--name N] # eval this
scripts/dummy-harness.sh down [--name N]--tls serves wss:// and an https:// WebUI with a throwaway self-signed
certificate for 127.0.0.1 (needs openssl); the instance's CID is then
wss:. Chromium's certificate interstitial takes thisisunsafe typed on the
page.
For a browser check, start it with --known-psk. The instance then uses
the operator PSK written in the script (dummy-not-a-secret), and up prints
the WebUI URL (env exports it as WEBUI_URL); hand that URL to Playwright
as it is. Without the flag the PSK is random, and building the URL means
reading a credential, which an agent's permission check refuses — it cannot
tell a dummy's secret from the live one. The known PSK costs nothing real: the
instance listens on loopback only and is gone on down.
Loopback only, ephemeral port, fresh operator secret, a fresh runner key
enrolled in the temp data dir before the server starts; down removes all of
it. The WebUI is served from webui/ on disk, not from the server binary's
embedded copy — that default is inverted here on purpose, because an embedded
WebUI is frozen at compile time and a JS/wasm edit would then show the OLD page
until make build plus a fresh instance. A stale page reads as "my change did
not work", which is the worst way to lose an afternoon. --agent claude runs a real claude (default model is the cheapest one) —
use it when the property under test involves a real agent's behaviour.
--agent fake emits claude's stream-json shapes on a timer, for checks that
must not depend on a model or the network.
Always drive it through eval "$(scripts/dummy-harness.sh env)" rather than
reading the state file yourself: the env output starts with unset lines that
matter (below).
Four traps, in the order you will hit them
1. Your shell's inherited HARNESS_AUTH_TICKET is for the LIVE server.
Every task shell has one. harness-cli prefers a ticket over the operator
secret, so pointing it at a dummy server fails with psk auth: psk: server
rejected: BadTicket — which reads like a secret mistake and sends you to check
the secret. The env
subcommand emits the unsets for this reason; if you construct the environment
by hand, scrub HARNESS_* yourself.
2. CLAUDE_CODE_* / CLAUDECODE / AI_AGENT are inherited too. A claude
spawned with those set decides it is a child session and silently disables
transcript saving, so --continue / --resume later find nothing. The script
scrubs them; the symptom if you don't is a banner reading
⚠ Transcript saving is off — inherited CLAUDE_CODE_CHILD_SESSION marker.
3. agent-runner's --max-tasks defaults to 1. An interactive session
holds its slot for its entire life, so the moment you open one, every follow-up
foreground open is refused runner_busy and every --background one sits
Queued forever. The second symptom is silence — no error anywhere, the task
simply never runs. ls shows tasks=1/1 and Busy; check that before
debugging anything else. The script passes
--max-tasks 4.
4. A restart's effect on in-flight tasks depends on HOW you stopped the
server. A deliberate stop — SIGTERM, or the --shutdown-file sentinel, as
scripts/restart.py and the dummy's own down use — holds them: the runner
keeps the children alive, the tasks read held between the two servers, and the
next server re-adopts them (a session comes back detached). A crash — kill -9 — recovers nothing and leaves the old
behaviour: Failed with err="runner_disconnected", or
err="server_restart" from the replay's own sweep.
So output stopping after a restart is no longer self-explanatory. Check ls
first: held means the mechanism is working and you are inside the window;
Failed err="hold_expired" means the window passed before a server came back;
Failed err="not_held_by_runner" means the runner reported nothing for it.
--hold-window=0 on the server restores the old kill-everything behaviour if
that is what a test needs.
Two traps specific to testing the hold itself: the window is 90s by default, so a debugging pause between the stop and the restart will expire it and look like a bug; and a held session's output resumes only after re-adoption, so a screen that stops at the restart is expected until the runner reconnects.
Testing a client across a server restart
The window you are watching through must not live on the server you are about to bounce. Two instances:
scripts/dummy-harness.sh up --detach --name host --agent fake
scripts/dummy-harness.sh up --detach --name target --agent fakeOpen an interactive bash session on host (every instance ships a bash
profile), start the client inside it pointed at target, then restart
target's server. The host session — and your snapshot/send access to it —
is untouched.
Inside that host session the shell still carries host's own HARNESS_*, so
unset them there before running a client against target:
session exec "$HSID" "unset HARNESS_AUTH_TICKET HARNESS_TASK_ID HARNESS_SERVER_CID; export HARNESS_OPERATOR_PSK='<target operator psk>'; echo ready"Restarting target's server means killing its pid and relaunching
harness-server with the same --listen port, --operator-psk and
--data-dir — the data dir holds the enrolled runner key — so the runner and
the client reconnect to what they think is the same server.
Driving harness-tui specifically
General TUI technique is in session-debugging — in particular, read the
reverse span out of snapshot --style instead of guessing where focus is.
Harness-specific facts:
- A detached session's PTY is
0 0until a client attaches. The size only ever arrives as aTerminalWindowSizeframe from an attached client, which reads its own terminal (objtrsf/exec/exec.go,winsize_unix.go); nobody sends one to a session you are only driving withsend/snapshot. Soharness-tuistarted that way dies instantly withterminal too small (need at least 80x24). Size it with
--resize, NOT withstty. This section saidsttyuntil 2026-08-24; it predatessession resizeand is wrong for anything you then intend to READ, because the server never learns a size set that way. The mechanism is insession-debuggingundersession resize— the harness- specific part is only thatharness-tuiis the program this bites hardest, since it refuses to start at all below 80x24:harness-cli session send --resize 45x200 --snapshot --settle-ms 3000 "$ID" ''- Once the TUI attaches to a session of its own, that nested PTY inherits the
TUI's size, so
stty sizeinside the attached shell is a free check that the handover really propagated the winsize. - Tab order is
<shown table>, notify, cmdresult, cmdline: the runners, tasks and schedules tables are ONE stop, switched with←/→(\x1b[D/\x1b[C) while a table has focus. Tab is+1, Shift-Tab (\x1b[Z) is-1. - Initial focus is
tasks. A reflexive Tab on startup moves focus off the panel you want. - Enter on the
taskspanel does nothing (it used to follow the task log);dopens the selected task's detail popup. A task's output is read withsession snapshot/ the grid view while it lives,transcript getorrecord getafterwards.
The bash profile is your agent-side hands
Every instance gets a bash profile alongside the agent one. A one-shot
session opened on it runs its prompt as a shell command (bash -c <prompt>)
with that task's HARNESS_* set, which is the only way to exercise
agent-side commands (harness-cli agent send, agent inbox, …) without
minting a ticket by hand:
harness-cli --server-cid "$CID" session new -d --oneshot --repo "$REPO" --agent bash \
--prompt "harness-cli agent send --topic chat.<first8-of-target-task> --data hello"The session ends when the command does, with its exit code. Its output is gone
once it ends unless you opened it with --record (record get <id>); read
session snapshot <id> while it runs, or make the command's result visible
some other way (an agent send, a file in the worktree). A long
--prompt "sleep 3600" keeps a task running with a worktree, for testing
forwards or file transfer against it.
That is also how you deliver an agentboard message to a session you are watching, since the wake fires on delivery.
The bin behind that profile is resolved, not the bare name, and it is shared
with the real bash runner preset (agent_presets.bash_bin()) so a dummy
cannot pass a check the thing it stands in for fails. It matters on Windows:
bare bash there is C:/Windows/System32/bash.exe, the WSL launcher, and a
task dispatched through it dies with execvpe(/bin/bash) failed inside a Linux
namespace holding none of the checkout. Git for Windows' bash is used instead.
On a machine where none is found, up says so on stderr and ships no bash
profile — so --agent bash is then rejected by name at open rather than
failing per task.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

