add-bats-test
Skill8 starsChanged 7 months agoApache-2.0View on GitHub
Every unit test file must follow this exact setup order: If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines. Do not call setup_mock_claude before _common_setup — the mock binary writes to $TEST_TMPDIR/mock_bin/ which doesn't exist until _common_setup creates it. Every agent must be skippable when findings exist and FORCE=false: For agents that should record costs:
In this repository22 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- .github/workflows/CLAUDE.mdCLAUDE.md
- lib/CLAUDE.mdCLAUDE.md
- prompts/CLAUDE.mdCLAUDE.md
- schemas/CLAUDE.mdCLAUDE.md
- scripts/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- add-edge-testSkill
- add-gather-agentSkill
- add-integration-testSkill
- add-synthesis-fixtureSkill
- audit-prompt-schema-syncSkill
- bundle-and-verifySkill
- check-ci-locallySkill
- debug-agent-failureSkill
- fixture-syncSkill
- pre-pr-checklistSkill
- rebuild-test-imageSkill
- release-ultrainitSkill
- run-testsSkill
- tune-synthesis-contextSkill
- validate-generated-artifactSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 32
Every unit test file must follow this exact setup order:
- Very specific: line 50
If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines.
- Very specific: line 78
Every agent must be skippable when findings exist and FORCE=false:
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Before You Start
tests/unit/agent_run.bats— canonical example of agent function teststests/unit/config_budget.bats— example of pure function tests (no mock claude)tests/helpers/test_helper.bash—_common_setup,make_claude_envelopeAPItests/helpers/mock_claude.bash—setup_mock_claude,create_dispatch_response
Steps
1. Determine test location and type
| Function under test | Test file location |
lib/*.sh function (no claude call) | tests/unit/<subsystem>.bats |
lib/*.sh function (calls run_agent) | tests/unit/<subsystem>.bats + mock claude |
| Phase entry function | tests/integration/<phase>.bats |
scripts/validate-*.sh | tests/scripts/validate_<name>.bats |
| Budget/state edge case | tests/edge/<domain>.bats |
2. Create the .bats file
Every unit test file must follow this exact setup order:
#!/usr/bin/env bats
# Tests for <function-name> in lib/<file>.sh
setup() {
load '../helpers/test_helper'
_common_setup # creates TEST_TMPDIR, exports defaults, sources all 9 libs
load '../helpers/mock_claude'
setup_mock_claude # installs mock binary on PATH (needed only if testing claude calls)
}
teardown() {
_common_teardown
}If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines.
Do not call setup_mock_claude before _common_setup — the mock binary writes to $TEST_TMPDIR/mock_bin/ which doesn't exist until _common_setup creates it.
3. Write the success path test
@test "<function>: writes findings on success" {
# Arrange: create mock response
make_claude_envelope \
"$(cat "$BATS_TEST_DIRNAME/../fixtures/findings/<agent>.json")" \
"0.15" > "$MOCK_CLAUDE_RESPONSE"
# Act
run run_agent "<name>" "@${SCRIPT_DIR}/prompts/<name>.md" \
"${SCRIPT_DIR}/schemas/<name>.json" \
"Read,Bash(find:*)" "haiku" "gather"
# Assert
assert_success
assert_file_exists "$WORK_DIR/findings/<name>.json"
run jq -e '.key' "$WORK_DIR/findings/<name>.json"
assert_success
}4. Write the resumability test
Every agent must be skippable when findings exist and FORCE=false:
@test "<function>: skips when findings exist and FORCE=false" {
# Pre-create findings
echo '{"findings":[]}' > "$WORK_DIR/findings/<name>.json"
FORCE=false run run_agent "<name>" "prompt" "schema" "tools" "haiku"
assert_success
# Verify mock was not called
run grep -c 'CALL:' "$MOCK_CLAUDE_LOG"
assert_output "0"
}5. Write the budget exhaustion test
@test "<function>: rejects when budget exhausted" {
# Pre-populate costs to exhaust budget
echo "gather|prior-agent|$(echo "$TOTAL_BUDGET + 1" | bc)" \
> "$WORK_DIR/costs/prior-agent.cost"
run run_agent "<name>" "prompt" "schema" "tools" "haiku"
assert_failure
}6. Cost file format assertion
For agents that should record costs:
@test "<function>: records cost to cost file" {
make_claude_envelope '{}' '0.42' > "$MOCK_CLAUDE_RESPONSE"
run run_agent "<name>" "prompt" "${SCRIPT_DIR}/schemas/<name>.json" "tools"
assert_success
assert_file_exists "$WORK_DIR/costs/<name>.cost"
run grep '0.42' "$WORK_DIR/costs/<name>.cost"
assert_success
}7. Important test conventions
- Set env vars BEFORE
_common_setup:config.shreads${VAR:-default}at source time. SettingTOTAL_BUDGETafter_common_setuphas no effect. - bc produces
.20not0.20for values < 1. Assert both forms:[[ "$x" == ".20" || "$x" == "0.20" ]] - Color codes:
_common_setupexports empty color codes (RED='') — ANSI codes are stripped for cleanassert_output --partialmatching. ULTRAINIT_MAX_RETRIES=1is set in_common_setup— tests won't wait for 3 retries.
Verify
make test-unitOr for a single file:
docker run --rm -v "$(pwd)":/workspace -w /workspace \
ghcr.io/joelbarmettleruzh/ultrainit-test:latest \
bats tests/unit/my-new-test.batsCommon Mistakes
- Passing raw fixture JSON without
make_claude_envelope—run_agentextracts.structured_outputfrom the response. Without the envelope wrapper, it writes"null"to findings silently. - Loading
mock_claudebefore_common_setup— the mock binary is written to$TEST_TMPDIR/mock_bin/which_common_setupcreates. Wrong order causessetup_mock_claudeto fail. - Not setting budget env vars before sourcing —
_common_setupsources all libs. Variables read at source time can't be changed after sourcing. Set overrides before calling_common_setup. - Using
assert_outputwithout--partialfor multi-line output — validator output includes dynamic metrics. Useassert_output --partial 'VERDICT: PASS'not exact match.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

