smoke-testing
Skill518 starsChanged 57 days agoMITView on GitHub
Use when deciding whether to run an out-of-process test suite (real binary or live instance) — picks between `make smoke`, `make abs-contract`, and `make predeploy-smoke` after the unit suite passes but you've changed handler wiring, scheduler jobs, downloader integration, ABS imports, or anything that boots the binary. For the routine pre-PR matrix (lint / vet / test / typecheck / build) see the `testing` skill.
- Reads credentials
We split this skill into the rules it gives an agent
4 rules. 0 are common, 0 are distinctive, and 4 are very specific to this project.
0
Common
Most of the 4 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 20
Default: make test is sufficient for most diffs.
- Very specific: line 53
Don't add new fixtures to tests/abscontract/ casually — they pin to upstream behaviour and exist to catch regressions, not to satisfy a …
- Very specific: line 54
Don't run make predeploy-smoke from a feature branch against production.
- Very specific: line 55
Don't commit a passing make smoke log if make test is failing — the smoke suite covers golden paths, not error paths.
“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Smoke testing Bindery
Bindery has three out-of-process test layers in tests/ that complement go test ./internal/.... Picking the wrong one wastes time; picking none and shipping wastes more.
Decision: which suite?
| Suite | Command | When to run | Cost |
| Unit + integration | make test | Every change. In-memory SQLite via db.OpenMemory. | ~2-3 min |
| Race detector | make test-race | Before opening a PR that touches concurrency: goroutines, background pools, shared caches. Sharded like CI. | ~45 min |
| HTTP smoke (real binary) | make smoke | Before opening a PR that touches handlers, routing, auth, scheduler wiring, embed, or cmd/bindery/. Boots the actual binary on a real port. | ~30–60s; requires make build first (the target chains it) |
| ABS contract | make abs-contract | Touching anything under internal/abs/, tests/abscontract/, or the ABS import flow. Pinned upstream contract — flakes here usually mean upstream drift, not your bug. | Up to 15 min |
| Predeploy | make predeploy-smoke | Only against a live instance. Requires BINDERY_URL and BINDERY_API_KEY env vars. Use after a deploy, not during development. | ~2 min, needs network |
Default: make test is sufficient for most diffs. Promote to make smoke when you've changed wiring, not just logic.
Before you run anything
# Smoke needs the binary built and the embedded SPA in place.
make build # equivalent: web-build → copy dist → go buildmake smoke chains make build automatically — but if you've only edited Go and want to skip the slow npm ci, build once and re-run go test -count=1 -timeout=60s ./tests/smoke/... directly.
Reading failures
- Smoke suite hangs at startup — usually the binary is panicking before it binds. Run
./binderydirectly and read stderr. - Smoke suite passes but unit tests fail — the in-memory and on-disk DBs disagree. Check for a missing migration or for a test that depends on real-clock timing.
- ABS contract reports diffs against the pinned snapshot — first verify it isn't upstream drift (
tests/abscontract/has fixture timestamps); only adjust the snapshot if the change is intentional. - Predeploy fails with 401/403 — the API key isn't being accepted. Check
X-Api-Keyheader reaches the server (reverse-proxy strip is the usual cause).
Pre-PR sequence
For changes touching the API surface, run, in order:
make lint
make test
make smoke
cd web && npm run typecheck && npm run build && cd ..Match what CI will run — see CONTRIBUTING.md §"Running the full local check suite" for the full matrix including govulncheck, golangci-lint v2.11.4 pinning, and the Docker buildx rehearsal.
Don't
- Don't add new fixtures to
tests/abscontract/casually — they pin to upstream behaviour and exist to catch regressions, not to satisfy a flaky run. - Don't run
make predeploy-smokefrom a feature branch against production. It hits real endpoints. - Don't commit a passing
make smokelog ifmake testis failing — the smoke suite covers golden paths, not error paths.
Other versions4 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 4
- kid-sid/claude-spellbook★ 190same name
- ncklrs/claude-chrome-user-testing★ 20same name
- fusebase-dev/fusebase-flow★ 9same name
- Bridev04/clauderail★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository10 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- autofixSkill
- commitsSkill
- discord-reportsSkill
- fix-issueSkill
- prsSkill
- tag-releaseSkill
- testingSkill
- triageSkill
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
Reviews can't be read right now.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.