harness-loop
Skill0 starsChanged 4 months agoApache-2.0View on GitHub
Test-drive the pyharness agent from the outside — run a probe task headlessly, inspect what happened with low context cost, fix the harness, re-run. Use when a change needs a real end-to-end run to verify (not just make test), when debugging why a session failed, or when asked to "try a run and see what happens".
- Reads credentials
5 other projects use a version of this.
Adopted by
Most recently taken up by
- niksavis/handilytheir own version, first seen Nov 8, 2026
- niksavis/basiclytheir own version, first seen Nov 7, 2026
- me-cedric/agent-compasstheir own version, first seen Nov 1, 2026
- anyoneanderson/agent-skillstheir own version, first seen Oct 13, 2026
- Chachamaru127/claude-code-harnesstheir own version, first seen Oct 13, 2026
Other versions5 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 5
- Chachamaru127/claude-code-harness★ 3.2ksame name
- anyoneanderson/agent-skills★ 3same name
- me-cedric/agent-compass★ 3same name
- niksavis/basicly★ 2same name
- niksavis/handily★ 2same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
The file
We split this skill into the rules it gives an agent
1 rule. 0 are common, 0 are distinctive, and 1 is very specific to this project.
0
Common
Most of the 4 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 52
Delete throwaway .sessions/run-* dirs when done if they clutter (they are gitignored either way).
“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
The harness loop: run → observe → fix → re-run
You are the coding agent working ON the harness; the pyharness agent runs INSIDE it. This loop lets you exercise a real session and read the results without flooding your context.
1. Probe
uv run pyharness run "<small task that exercises the change>" --json- Needs
ANTHROPIC_API_KEYin.env(a real run costs real money —make testdoes not and stays the default verification). Cap spend with--budget 1for cheap probes. - Each run gets a fresh
.sessions/run-<timestamp>dir; the digest lands on stdout as one JSON object, everything else on stderr. - Approvals are denied by default (no stdin). If the probe legitimately
needs gated actions (skill writes, package installs, mutating HTTP), add
--approve-all— it approves per-call and never mints grants. - Exit code = outcome: 0
answered, 2stopped:max_steps, 3stopped:budget, 4error, 5aborted/empty.
2. Observe — cheapest view first
uv run pyharness show # digest of the latest session (no API key)
uv run pyharness show <name> --transcript # what it actually did, flattened
uv run pyharness-index --sql "SELECT name, outcome, cost_usd FROM sessions ORDER BY started DESC LIMIT 10"The digest answers "did it work, what did it cost, was anything denied". The
transcript answers "what did it do" — task, agent text, code, outputs, errors,
skill uses, answer. Never cat trace.jsonl: every llm_call entry embeds
the full prompt snapshot, so the raw file is orders of magnitude bigger than
the session; if the transcript isn't enough, filter one event kind, e.g.
jq -c 'select(.kind=="action_start")' <dir>/trace.jsonl. The audit trail
(history of capability calls, denials) is <dir>/audit.jsonl, and
make verify-audit DIR=<dir> checks its hash chain.
For watching a probe live: the run prints a [watch] URL on stderr
(http://127.0.0.1:6061), and make watch tails .sessions/ from outside.
3. Fix and re-run
Fix the harness code, run make test until green, then repeat the probe.
Delete throwaway .sessions/run-* dirs when done if they clutter (they are
gitignored either way).
Boundary — do not widen it
You (the coding agent) may modify the whole harness. The in-session agent may
not: its self-improvement surface is skills/lessons only (~/.pyharness/skills
behind an approval gate), its file writes are jailed to the session workspace,
and the OS sandbox blocks writes to this repo. Keep it that way — never "fix" a
probe failure by loosening policy defaults, the workspace jail, or the sandbox.
A denial in the digest is usually the harness working as designed.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

