building-and-testing
Skill83 starsChanged 2 days agoMITView on GitHub
Build, test suite, CI status, build/test failures - make build, test-gpu, verify-fast, GTEST_FILTER, Docker/CUDA toolchain, dependency bumps, "CI is red", "which gate failed", stale objects / segfault after a header edit, hooks, docker-entrypoint env vars, determinism, perplexity. Not perf (benchmark-cuda), not output quality (check-degeneration).
- Reads credentials
- Deletes or force-pushes
- Commits and pushes
1 other project uses a version of this.
Adopted by
Most recently taken up by
- quarkusio/quarkustheir own version, first seen Oct 8, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- quarkusio/quarkus★ 16ksame name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository16 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- add-model-archSkill
- benchmark-cudaSkill
- check-degenerationSkill
- codebase-auditSkill
- code-graphSkill
- docs-layersSkill
- docs-syncSkill
- find-stubsSkill
- numerics-bisectSkill
- quant-formatsSkill
- server-apiSkill
- shipping-prsSkill
- sm120-cuda-expertSkill
The file
Files
Build & Test - imp
Hard rules
| # | Rule | Detail |
| 1 | No CUDA toolkit on the host | Everything runs in Docker. Toolchain nvidia/cuda:13.5.0-devel-ubuntu26.04, GCC 15.2, C++23 for host AND device (CMAKE_CUDA23_STANDARD_COMPILE_OPTION shim at the top of CMakeLists.txt). |
| 2 | build/ and build-dev/ are root-owned | make dev-clean, or docker run --rm -v $PWD:/src -w /src ubuntu rm -rf build. Never sudo. |
| 3 | No --mount=type=cache on the build dir in the Dockerfile | 03a2cc19: ninja reused stale objects. The ccache mount (id=imp-ccache) is content-addressed and stays; docker builder prune empties it, which only costs one cold build. |
| 4 | models/ in the repo is a symlink farm | Custom docker run mounts $HOME/models:/models; Makefile targets already do. |
| 5 | Dependency pins live ONLY in cmake/imp-deps.cmake | CUTLASS, GTest, httplib, nlohmann/json, each a TAG plus the SHA the build fetches. make build injects both via scripts/dep_build_args.sh. check_dep_pins.sh runs offline in the deps gate (drift, missing SHA, unpinned uses:) and --online in Lint (tag must still resolve to the pinned commit). |
| 6 | CI has no GPU runner | Test job skipped unless repo var HAS_GPU_RUNNER=true. GPU correctness and perf are gated LOCALLY, by hand or with IMP_HOOK_GPU=1 on the hooks: pre-commit (make build + GPU modules or the full make test-gpu), pre-push (make verify-fast, plus the paired A/B make verify-ab on kernel diffs). The hook prints the skipped command; run it before the PR. |
| 6a | git commit / git push are CPU jobs | Both hooks run make preflight only. With IMP_HOOK_GPU=1 they and make GPU targets refuse while another worktree holds scripts/gpu_lock.sh (scripts/require_free_gpu.sh, hook_gpu in scripts/hook_lib.sh), then check load. Holding the card across jobs: scripts/gpu_lock.sh acquire "<purpose>" in your worktree, release after, status shows the holder (lock /tmp/gpu-lock, shared with ~/.claude/skills/gpu-stats/gpu-lock.sh, keeper TTL 4 h; a live pid in the old /tmp/imp-gpu.lock also refuses). Still send the START/END handshake to other sessions. No --no-verify: the hooks touch no GPU by default. The skipped GPU tests are owed: run them and report the result in the PR (CI Test never runs: .github/workflows/ci.yml:999). |
| 7 | Required check = Build, ruleset 14716423 | Static gates run as its FIRST step (scripts/ci_static_gates.sh, unfiltered) and block the merge since #1527. Rename the job without the ruleset and every PR sits at mergeStateStatus=BLOCKED. |
| 8 | main's CI status is stale by default | Auto-merge squashes with GITHUB_TOKEN, which starts no workflow run. Judge a fix by the PR run or the file; refresh with gh workflow run CI --ref main. |
Command table
| Task | Command | Time |
| Incremental build (inner loop) | make dev | 2-14 s |
| Incremental build + CI unit lane | make dev-test (= ctest -L unit) | ~3 s |
| Full image (the gate; anything you measure or push) | make build -> imp:test (main checkout) or imp:test-<dir>-<hash8> (worktree, scripts/image_tag.sh) | ~6 min cold; 10-40 s from a warm ccache (595 TUs, 2026-09-11); 0 s when the image already carries the tree (imp.tree label) |
| CPU unit binary | make test-unit (imp-tests-unit) | <5 s |
| Full GPU suite | make test-gpu | 4-10 min |
| E2E on real models | make test-e2e | SafeTensors only: Qwen3-8B NVFP4, Qwen3.8-27B NVFP4, Gemma-4-26B-A4B NVFP4 (also MOE_MODEL, the determinism suite) (paths in Makefile) |
GDN restore group of test-e2e (a scripts/check-release.sh stage) | make test-hybrid-restore | Qwen3.5-4B-mxfp4; a skip fails the stage |
| Server e2e (the ONLY place handlers/batching run end to end) | make test-server | includes degen_suite.py and tests/test_server_tracing.py |
| Rerank / agents / external harnesses | make test-rerank, make test-agents, make test-agents-external | GPU + local model |
| NIAH retrieval gate, spec fidelity | make test-niah, make test-spec-fidelity | spec-fidelity needs ~26 GB free |
| Vision goldens | make test-vision (IMP_VISION_GOLDEN_DUMP=1 regenerates) | |
| Chat-template goldens | make chat-goldens (network, writes tests/refs/chat_template_goldens.h) | |
| Pre-push gate | make verify-fast | 18 s without perf gate, 36 s with |
| Full / chunked / north-star gates | make verify, make verify-chunked, make verify-north-star | ~5 min full |
Kernel register/spill ratchet (CI kernels gate) | make kernel-resources, re-pin make kernel-resources-update | needs libimp.a |
| clang-format | make format-check; never bare make format | repo is not format-clean |
| clang-format, changed lines only (what CI checks) | per file: ranges from git diff -U0 main -- f, then docker run --rm --user $(id -u):$(id -g) -v $PWD:/work -w /work $CLANG_FORMAT_IMG clang-format --style=file -i --lines=a:b ... f (--dry-run to check) | image name: CLANG_FORMAT_IMG in Makefile |
| Host ASan/UBSan over test-core/test-text | make asan | works on WSL2 |
| compute-sanitizer | make sanitize | does NOT work on WSL2 (WDDM) |
| Alloc-interpose census | make check-alloc-interpose (build-interpose/) | never benchmark it (reads ~3% low) |
| Server gcov | make coverage | GPU + model |
| API mock contract | pytest tests/api/ or tests/api/run_mock_tests.sh | no GPU; the mock lane also runs the nomodel tests |
| GPU free? | make check-gpu (scripts/require_free_gpu.sh: utilisation AND memory) |
Filtered GPU run (GTEST_FILTER is not threaded through make test-gpu):
docker run --rm --gpus all -v $HOME/models:/models \
-e IMP_TEST_MODEL=/models/Qwen3-8B-Q8_0.gguf \
imp:test imp-tests --gtest_filter="DegenerationTest.*"DetEvalE2ETest.* matches 0 tests and prints PASSED; the working filter is *DetEvalE2ETest*.
Binaries
| Where | What |
imp:test image | imp-server, imp-cli, imp-bench, imp-quantize, imp-tests (full GPU), imp-tests-unit (generated wrapper = ctest -L unit: test-core + test-text + e2e unit filter), modules test-core test-text test-compute test-attention test-quant test-kv test-moe-gdn test-e2e |
build-dev/ (host, from make dev) | same binaries; run them inside imp:toolchain with -v $PWD:/src -w /src (that mount hides the image's own /src) |
tests/test_ssm.cu -> test-moe-gdn. SamplingTest -> test-compute. A new CPU test belongs in test-core. Lane census: python3 tools/check_test_lanes.py --report (the Test lanes check fails on a test in no ctest entry or a unit-lane test demoted vs the base; a new GPU test in a gpu module needs no edit). The unit lane has no skips (guard_unit_skips in ctest -L unit fails on any): a test that needs a GPU or a model file goes to test-kv or to test-e2e outside _unit_e2e_filter; fixture files live in tests/fixtures/ (IMP_TEST_FIXTURES_DIR). A new tool binary needs a cp in the builder stage AND a COPY --from=builder line in the Dockerfile.
Gates and hooks
| Gate group | Runs where | Content |
filesize | Build, hooks | check_filesize.py ([allow] pin = ceiling, +25 slack, shrink free), check_determinism_sites.py, check_dead_inline_accessors.py, check_log_fatal.py |
lanes | Build, hooks, own check Test lanes (#1770) | check_test_lanes.py --report (macros per lane, no-lane delta vs base); guard_unit_skips inside ctest -L unit fails on any runtime skip |
entrypoint | Build, hooks | tests/test_entrypoint.sh drives docker-entrypoint.sh against a stub (25 assertions) |
alloc | Build, hooks | check_alloc_sites.py + check_alloc_pairs.py (allowlist is two-way: removing a site needs the allowlist edit) |
kernels | Build only (needs artifact) | make kernel-resources vs tools/kernel_resource_baseline.txt (REG >= 240 or non-zero local frame) |
launchguards | Build, hooks | check_launch_guards.py |
docs | Build, hooks, Docs job | sync_docs.py --check, docs_lint.py |
citations | Build, hooks | check_doc_citations.py over all living docs (#1783) |
hygiene | Build, Release hygiene job only | check-release.sh with SKIP_VERIFY=1 (also resolves relative links in docs) |
Advisory jobs: Lint, clang-tidy, Mock API contract, Real API contract (model-less), Sanitizers, PTX fallback.
- Hooks:
make install-hookscopiesscripts/pre-commit.hookandscripts/pre-push.hookintogit rev-parse --git-path hooks(shared by all worktrees). Never edit.git/hooks/*:guard_precommit_filterin the CPU lane diffs them against the tracked files. make buildrunsscripts/build_image.sh: skipped when the worktree image already carries the tree and build args (labelsimp.tree=scripts/image_tag.sh tree, unstaged and untracked files included;imp.build;imp.worktree).IMP_FORCE_BUILD=1builds anyway. Runners refuse an image whoseimp.treeis not this tree (IMP_ALLOW_FOREIGN_IMAGE=1overrides). Saves 3.5 min per hook stage on one tree.- Pre-commit =
make preflight(~20 s); GPU step withIMP_HOOK_GPU=1only:make build, then only the test modules the staged diff reaches (modules_forinscripts/pre-commit.hook:src/memory|runtime-> test-core test-kv test-e2e,src/compute-> test-compute test-attention test-quant test-kv test-moe-gdn,src/exec-> test-e2e test-moe-gdn test-compute test-attention,src/model|quant-> test-quant test-text test-e2e test-core, atests/file -> its module); the fullmake test-gpu(3108 tests inimp-tests --gtest_list_tests, 2026-09-28) oninclude/ src/core src/api CMakeLists cmake/changes or withIMP_PRECOMMIT_FULL=1..md .py .hookskip. - Pre-push =
make preflight; GPU step withIMP_HOOK_GPU=1only:require_free_gpu.sh, thenmake verify-fast; the perf gate runs only when the diff matchesPERF_RE(src/{compute,exec,quant,runtime,model}/, any.cu/.cuh,cmake/,CMakeLists.txt,tests/perf_baseline*). Onsrc/computeorsrc/execdiffs the paired A/B (make verify-ab, 2 %, ~5 min) replaces the single-arm perf gate;IMP_SKIP_AB=1falls back to it. Skips:IMP_VERIFY_SKIP_PERF/_VRAM/_GRAPHS=1. - Hooks run
make preflight: everyci_static_gates.shgroup,hygieneincluded. docs_lint.pywritesdocs/audit/docs-rewrite/STALE.mdonly with--write-stale; hooks and CI never touch it (#2194).
Traps
| Symptom | Cause | Fix |
Segfault / cudaFree on garbage / MoEExecutorTest 5/5 red after a header or class-layout edit | stale .o despite ninja | grep -rl exec/executor.h src tools tests | xargs touch && make dev (healed 3/3) |
Link error __cudaRegisterLinkedBinary after make dev | same class | touch includers, rebuild |
| Gate result contradicts the tree | make build COPYs the tree at start (#1531); verify-fast never builds (SKIP build line) | rebuild after every edit or amend, then gate |
| Bench numbers move while a chain runs | make dev rebuilds build-dev/ under the running binary | bench from copies (-v dir:/bin_arm) |
NvFP4SmallMTest.BandwidthAboveStarvationFloor red in the full suite | contention in the run (401 vs 537.6 GB/s threshold; isolated 590-622) | rerun isolated |
NvFP4SmallMV2Test.BandwidthAboveStarvationFloor red in the pre-commit suite | same class: 39.68 % vs the 40 % floor, isolated 73-78 % (3/3, 2026-09-22) | rerun isolated (docker run --rm --gpus all imp:test test-quant --gtest_filter=...); only with IMP_HOOK_GPU=1 does the hook run it |
ci_static_gates.sh in imp:toolchain: FAIL registers + local frame vs the pin, docker: command not found | the kernels group shells out to docker | run make kernel-resources on the host (reads build-dev/libimp.a: make dev first) |
test-core fails to link in Sanitizers / make asan | sources inside the IMP_BUILD_SERVER block (#1821) | keep test-core's deps outside that block |
--perplexity OOMs or pool floors on a head checkpoint | mtp_k=auto loads the MTP head (+0.79 GiB) | --set speculative.mtp_k=0 in PPL harnesses |
| PPL differs 0.35% between runs of the SAME binary | non-deterministic forward | --set runtime.deterministic=true on both arms (implies deterministic_gemm) |
std::sort etc. "not declared" | libstdc++ 15 dropped transitive <algorithm> (#906) | add the include |
| nvcc arch error | target is compute_120a/sm_120a + optional compute_120f PTX (IMP_SM120_FLAGS) | never generic sm_120 (sm120-cuda-expert) |
| Compose/container ignores a config key | only IMP_CONFIG / IMP_SET reach every key (#1823); the 19 legacy IMP_* names in docker-entrypoint.sh are frozen | -e IMP_SET="key=v key2=v"; a bogus key fails at start (no such key) |
| Value looks like an unread env var | engine reads only IMP_CONFIG IMP_DETERMINISTIC IMP_FMHA_FA2 IMP_WORKER_TIMING IMP_SPEC_TRACE IMP_PPL_DUMP; the entrypoint translates the rest | grep docker-entrypoint.sh too |
git worktree remove fails | root-owned build-dev/ inside it | rm -rf via container, git worktree prune; a worktree also needs the models/ symlink dir or verify-fast fails in the hook |
FAIL model directory not found: /src/models from make verify-fast in a git worktree | models/ symlink farm exists only in the main checkout (scripts/verify.sh:192) | IMP_VERIFY_MODELS=$HOME/models make verify-fast (or ln -s $HOME/models models) |
A gitignored ./imp.conf in the repo root | dev builds with -w /src load it, images do not (#1784 fixed the build context) | read the imp.conf loaded from line in every log |
Determinism and quality
--set runtime.deterministic=true: temp=0 reproducibility incl. MoE routing atomics and top-k races; impliesruntime.deterministic_gemm; promoted process-wide since #542; default OFF. Qwen3.6-35B stays non-deterministic at temp=0 (routing flips); never assert exact output there.- 7
DISABLED_tests with reasons: 2 determinism boundaries (cross-context GDN, FMHA smem), the rest benches without assertions. Do not re-enable. imp-cli --perplexity <file>is teacher-forced prefill (chunk-aware since #553); usetools/analysis/ppl_corpus_45k.txt(untracked, rebuild withtools/analysis/make_ppl_corpus.sh), not the 199-tokenppl_corpus.txt.imp-cli --json: one JSON document on stdout, logs on stderr (#1715).
PR / merge
Ship flow, auto-merge race, release cut: skill shipping-prs.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

