test-regression
Skill1 starsChanged 38 days agoMITView on GitHub
Pre-commit verification gate for markdown-memory - runs ruff (lint and format check), mypy --strict, pytest and the live end-to-end test in sequence, stopping at the first failure. Use before committing or opening a PR, after finishing a change, or when asked to "run all checks", "verify everything passes", or "is this ready to commit".
1 other project uses a version of this.
Adopted by
Most recently taken up by
- moah0911/aftermaththeir own version, first seen Oct 25, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- moah0911/aftermath★ 14same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository4 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- reindex-docsSkill
- run-evalSkill
The file
Files
test-regression
Run
scripts/check.sh # licences, ruff, ruff format, mypy --strict, pytest, live_test.py
scripts/check.sh --fast # same without the live end-to-end testThe script is fail-fast (set -euo pipefail): the first failing step stops the run and its
exit code is non-zero. A full run takes about two minutes; most of that is the real
embedding model loading in pytest and in the live test.
The steps
| Step | Command | Passing looks like |
| Licences | uv run python scripts/fetch_eval_corpus.py --check | corpus matches the manifest |
| Lint | uv run ruff check . | All checks passed! |
| Format | uv run ruff format --check . | N files already formatted |
| Types | uv run mypy --strict src/ | Success: no issues found |
| Tests | uv run pytest -q | all passed (tests marked embedding skip only if the model cannot load) |
| End-to-end | uv run python scripts/live_test.py | LIVE TEST PASSED: N checks |
The live test spawns the real server as a subprocess and drives it over stdio JSON-RPC: tool discovery, indexing, outline, exact section boundaries, keyword and semantic search, error reporting, incremental re-indexing, stdout hygiene, schema integrity, server memory (< 3 GB peak) and warm latency.
On failure
- Report the failing step and its output verbatim. Do not summarise an error away.
- Fix the cause, not the check: no
# type: ignore,# noqa, skipped tests, loosened assertions or raised thresholds to get to green. If a check is genuinely wrong, say so and let the user decide. uv run ruff check . --fixanduv run ruff format .may be applied freely; re-run the gate afterwards.- Re-run the whole gate after a fix, not just the step that failed.
Not covered here
Changes to ranking, chunking, unit extraction or embedding also need the retrieval gate -
use the run-eval skill (uv run python scripts/eval_retrieval.py). It is separate because
it measures search quality rather than correctness.
Committing is a separate, explicit step: a green gate is not permission to commit.
Two places run this same gate. .githooks/pre-push runs it locally once the clone has been
pointed at it (git config core.hooksPath .githooks), and .github/workflows/gate.yml runs
the same steps on every pull request and on pushes to main, across Python 3.11 to 3.14.
The hook is
the authoritative one to satisfy - it is what you can run - but it is skippable with
--no-verify, which is exactly why CI exists. A test holds the two step lists in the same
order, so they cannot drift apart.
Running it by hand first is still faster than finding out at push time.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.