agentleFS
joelbarmettlerUZHjoelbarmettlerUZH/ultrainit.sh/.claude/skills/add-bats-test/SKILL.md

add-bats-test

Skill8 starsChanged 7 months agoApache-2.0View on GitHub

Every unit test file must follow this exact setup order: If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines. Do not call setup_mock_claude before _common_setup — the mock binary writes to $TEST_TMPDIR/mock_bin/ which doesn't exist until _common_setup creates it. Every agent must be skippable when findings exist and FORCE=false: For agents that should record costs:

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every unit test file must follow this exact setup order:

    line 32
  2. Very specific:

    If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines.

    line 50
  3. Very specific:

    Every agent must be skippable when findings exist and FORCE=false:

    line 78

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Before You Start

  • tests/unit/agent_run.bats — canonical example of agent function tests
  • tests/unit/config_budget.bats — example of pure function tests (no mock claude)
  • tests/helpers/test_helper.bash — _common_setup, make_claude_envelope API
  • tests/helpers/mock_claude.bash — setup_mock_claude, create_dispatch_response

Steps

1. Determine test location and type

Function under testTest file location
lib/*.sh function (no claude call)tests/unit/<subsystem>.bats
lib/*.sh function (calls run_agent)tests/unit/<subsystem>.bats + mock claude
Phase entry functiontests/integration/<phase>.bats
scripts/validate-*.shtests/scripts/validate_<name>.bats
Budget/state edge casetests/edge/<domain>.bats

2. Create the .bats file

Every unit test file must follow this exact setup order:

#!/usr/bin/env bats
# Tests for <function-name> in lib/<file>.sh

setup() {
    load '../helpers/test_helper'
    _common_setup              # creates TEST_TMPDIR, exports defaults, sources all 9 libs
    load '../helpers/mock_claude'
    setup_mock_claude          # installs mock binary on PATH (needed only if testing claude calls)
}

teardown() {
    _common_teardown
}

If the functions under test never call run_agent or claude -p directly, omit the mock_claude lines.

Do not call setup_mock_claude before _common_setup — the mock binary writes to $TEST_TMPDIR/mock_bin/ which doesn't exist until _common_setup creates it.

3. Write the success path test

@test "<function>: writes findings on success" {
    # Arrange: create mock response
    make_claude_envelope \
        "$(cat "$BATS_TEST_DIRNAME/../fixtures/findings/<agent>.json")" \
        "0.15" > "$MOCK_CLAUDE_RESPONSE"

    # Act
    run run_agent "<name>" "@${SCRIPT_DIR}/prompts/<name>.md" \
        "${SCRIPT_DIR}/schemas/<name>.json" \
        "Read,Bash(find:*)" "haiku" "gather"

    # Assert
    assert_success
    assert_file_exists "$WORK_DIR/findings/<name>.json"
    run jq -e '.key' "$WORK_DIR/findings/<name>.json"
    assert_success
}

4. Write the resumability test

Every agent must be skippable when findings exist and FORCE=false:

@test "<function>: skips when findings exist and FORCE=false" {
    # Pre-create findings
    echo '{"findings":[]}' > "$WORK_DIR/findings/<name>.json"

    FORCE=false run run_agent "<name>" "prompt" "schema" "tools" "haiku"
    assert_success

    # Verify mock was not called
    run grep -c 'CALL:' "$MOCK_CLAUDE_LOG"
    assert_output "0"
}

5. Write the budget exhaustion test

@test "<function>: rejects when budget exhausted" {
    # Pre-populate costs to exhaust budget
    echo "gather|prior-agent|$(echo "$TOTAL_BUDGET + 1" | bc)" \
        > "$WORK_DIR/costs/prior-agent.cost"

    run run_agent "<name>" "prompt" "schema" "tools" "haiku"
    assert_failure
}

6. Cost file format assertion

For agents that should record costs:

@test "<function>: records cost to cost file" {
    make_claude_envelope '{}' '0.42' > "$MOCK_CLAUDE_RESPONSE"
    run run_agent "<name>" "prompt" "${SCRIPT_DIR}/schemas/<name>.json" "tools"
    assert_success
    assert_file_exists "$WORK_DIR/costs/<name>.cost"
    run grep '0.42' "$WORK_DIR/costs/<name>.cost"
    assert_success
}

7. Important test conventions

  • Set env vars BEFORE _common_setup: config.sh reads ${VAR:-default} at source time. Setting TOTAL_BUDGET after _common_setup has no effect.
  • bc produces .20 not 0.20 for values < 1. Assert both forms: [[ "$x" == ".20" || "$x" == "0.20" ]]
  • Color codes: _common_setup exports empty color codes (RED='') — ANSI codes are stripped for clean assert_output --partial matching.
  • ULTRAINIT_MAX_RETRIES=1 is set in _common_setup — tests won't wait for 3 retries.

Verify

make test-unit

Or for a single file:

docker run --rm -v "$(pwd)":/workspace -w /workspace \
    ghcr.io/joelbarmettleruzh/ultrainit-test:latest \
    bats tests/unit/my-new-test.bats

Common Mistakes

  1. Passing raw fixture JSON without make_claude_envelope — run_agent extracts .structured_output from the response. Without the envelope wrapper, it writes "null" to findings silently.
  2. Loading mock_claude before _common_setup — the mock binary is written to $TEST_TMPDIR/mock_bin/ which _common_setup creates. Wrong order causes setup_mock_claude to fail.
  3. Not setting budget env vars before sourcing — _common_setup sources all libs. Variables read at source time can't be changed after sourcing. Set overrides before calling _common_setup.
  4. Using assert_output without --partial for multi-line output — validator output includes dynamic metrics. Use assert_output --partial 'VERDICT: PASS' not exact match.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.