run-tests
Skill8 starsChanged 7 months agoApache-2.0View on GitHub
The Makefile targets use Docker. To run one file, use the docker command directly: Add --tap for TAP format or --verbose-run for assertion details: Integration tests always show green in CI even when failing — they run with || true in .github/workflows/test.yml. After make test-integration, read the output explicitly. Add echo "TMPDIR: $TEST_TMPDIR" >&3 inside the failing @test body (bats redirects >&3 to the terminal). Run the test in verbose mode. Look for CALL: lines. If the mock was never…
50 other projects use a version of this.
Adopted by
Most recently taken up by
- tatsoft-llc/FrameworX-Projectstheir own version, first seen Nov 15, 2026
- wisterx-spec/agent-railstheir own version, first seen Nov 15, 2026
- utopia-rise/fmod-gdextensiontheir own version, first seen Nov 9, 2026
- Pubx-ai/skillstheir own version, first seen Nov 8, 2026
- coralogix/cx-clitheir own version, first seen Nov 8, 2026
- DataDog/dd-trace-pytheir own version, first seen Nov 1, 2026
Used with
Projects with it also use prepare-release, create-pr and 3 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
ShopifySkillprepare-release
Prepare the changeset-release/main branch for release by syncing version.ts files and running pnpm install
Also in 3 projects that use this
SentrySkillcreate-pr
Create pull requests following Sentry conventions. Use when opening PRs, writing PR descriptions, or preparing changes for review. Follows Sentry's code review guidelines.
Also in 4 projects that use this
SentrySkillcommit
Create commit messages following Sentry conventions. Use when committing code changes, writing commit messages, or formatting git history. Follows conventional commits with Sentry-specific issue references.
Also in 4 projects that use this
MicrosoftSkillrelease
Use ONLY when the user explicitly asks to cut/publish a Huabu desktop release or tag a version. Performs an outward action — bumps the version in apps/desktop/package.json, commits + annotated-tags vX.Y.Z, and pushes the branch + tag. That tag push auto-builds the macOS/Windows installers and publishes a GitHub Release. Provides `status`, `bump`, `tag-push`, and a one-command `run` (bump → tag-push).
Also in 5 projects that use this
GoogleSkillcode-review
Adversarial systems, concurrency, memory-model, and architecture code review protocol. Use this skill whenever asked to review code, PRs, diffs, or branches in Filament.
Also in 4 projects that use this
Other versions50 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 50
- netbox-community/netbox★ 22ksame name
- yagop/node-telegram-bot-api★ 9.2ksame name
- gitextensions/gitextensions★ 8.6ksame name
- ethereum/consensus-specs★ 4ksame name
- dotnet/sdk★ 3.3ksame name
- ComplianceAsCode/content★ 2.8ksame name
- idaholab/moose★ 2.4ksame name
- CliMA/Oceananigans.jl★ 1.5ksame name
- serenedb/serenedb★ 1.1ksame name
- microsoft/testfx★ 1.1ksame name
40 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository22 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- .github/workflows/CLAUDE.mdCLAUDE.md
- lib/CLAUDE.mdCLAUDE.md
- prompts/CLAUDE.mdCLAUDE.md
- schemas/CLAUDE.mdCLAUDE.md
- scripts/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- add-bats-testSkill
- add-edge-testSkill
- add-gather-agentSkill
- add-integration-testSkill
- add-synthesis-fixtureSkill
- audit-prompt-schema-syncSkill
- bundle-and-verifySkill
- check-ci-locallySkill
- debug-agent-failureSkill
- fixture-syncSkill
- pre-pr-checklistSkill
- rebuild-test-imageSkill
- release-ultrainitSkill
- tune-synthesis-contextSkill
- validate-generated-artifactSkill
The file
We split this skill into the rules it gives an agent
4 rules. 0 are common, 0 are distinctive, and 4 are very specific to this project.
0
Common
Most of the 45 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 59
Add --tap for TAP format or --verbose-run for assertion details:
- Very specific: line 79
Add echo "TMPDIR: $TEST_TMPDIR" >&3 inside the failing @test body (bats redirects >&3 to the terminal).
- Very specific: line 87
Look for CALL: lines. If the mock was never called when it should have been, check if setup_mock_claude was called in setup().
- Very specific: line 107
Only needed when Dockerfile.test changes:
“Common” is measured: kept by at least 6 in 10 of the 45 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Before You Start
Makefile— all test targets and their Docker invocationstests/helpers/test_helper.bash—_common_setup,make_claude_envelopetests/helpers/mock_claude.bash—setup_mock_claude, dispatch modeDockerfile.test— the test image (must be built if changed)
Test Tiers
| Command | What it tests | Speed |
make test-unit | Individual lib/*.sh functions | ~30s |
make test-scripts | scripts/validate-skill.sh, scripts/validate-subagent.sh | ~15s |
make test-edge | Budget exhaustion, corrupt state, special chars, numeric edge cases | ~20s |
make test-integration | Phase-level pipeline (gather, synthesize, resume) | ~60s |
make test-all | All of the above | ~2min |
Running Tests
Full suite
make test-allSingle tier
make test-unit # fastest, run after every lib change
make test-scripts # after changes to scripts/validate-*.sh
make test-edge # after changes to budget, config, or CLI arg handling
make test-integration # after changes to gather.sh, synthesize.sh, or merge.shSingle .bats file
The Makefile targets use Docker. To run one file, use the docker command directly:
docker run --rm \
-v "$(pwd)":/workspace \
-w /workspace \
ghcr.io/joelbarmettleruzh/ultrainit-test:latest \
bats tests/unit/agent_run.batsVerbose output
Add --tap for TAP format or --verbose-run for assertion details:
docker run --rm -v "$(pwd)":/workspace -w /workspace \
ghcr.io/joelbarmettleruzh/ultrainit-test:latest \
bats --verbose-run tests/unit/agent_run.batsInterpreting Output
ok N - test name= passnot ok N - test name= fail, followed by assertion details# (from function ... source ... line N)= exact location of failed assertion
Integration tests always show green in CI even when failing — they run with || true in .github/workflows/test.yml. After make test-integration, read the output explicitly.
Debugging a Failing Test
1. Find the tmpdir
Add echo "TMPDIR: $TEST_TMPDIR" >&3 inside the failing @test body (bats redirects >&3 to the terminal). Run the test in verbose mode.
2. Inspect mock logs
cat $TEST_TMPDIR/.ultrainit/mock_claude.log # all mock invocationsLook for CALL: lines. If the mock was never called when it should have been, check if setup_mock_claude was called in setup().
3. Check dispatch matching
Dispatch mode matches agent name from the --json-schema argument basename (case-insensitive substring). If docs matches before docs-scanner, rename the dispatch file: longer/more-specific names first alphabetically.
4. Verify envelope wrapping
Passing raw fixture JSON where an envelope is expected causes run_agent to extract .structured_output as null, writing "null" to the findings file silently:
# Wrong — raw JSON, no envelope
cat tests/fixtures/findings/identity.json > "$MOCK_CLAUDE_RESPONSE"
# Correct — envelope-wrapped
make_claude_envelope "$(cat tests/fixtures/findings/identity.json)" "0.15" > "$MOCK_CLAUDE_RESPONSE"Building the Test Image
Only needed when Dockerfile.test changes:
make test-image # builds locallyThe CI image is pulled from ghcr.io/joelbarmettleruzh/ultrainit-test:latest. If the image doesn't exist yet on a fresh fork, run make test-image first.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

