test-strength
Skill790 starsChanged 4 months agoMITView on GitHub
Measure whether Python tests detect behavior changes through diff-scoped mutation testing. Use when Codex needs to audit the strength of a pytest suite, evaluate whether tests cover changed code, investigate surviving mutants, or propose and verify targeted tests for missed behavior.
In this repository9 other files it gives its agents.
- crazy-ecommerce-builderSkill
- customer-finderSkill
- html-game-generatorSkill
- landing-page-generatorSkill
- launchauditSkill
- premium-ui-revampSkill
- react-screenshot-recreatorSkill
- startupblueprintSkill
- talking-avatarSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 4 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Run the bundled runner from the repository being assessed.
- Distinctive: line 13
Run the bundled script, using its absolute skill path:
- Distinctive: line 30
Verify that test before suggesting it:
- Distinctive: line 41
Use this structure and include only real survivors:
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test Strength
Run the bundled runner from the repository being assessed. Keep the repository working tree unchanged: the runner copies it to a temporary directory before each mutant run.
Run the audit
- Confirm that Python and
pytestare available. Run from the target repository root. Run the bundled script, using its absolute skill path:
python <skill-path>\scripts\strength.py [--base <ref>] [path]Omit both arguments to compare the current worktree with the merge-base of the default branch. Use
--base main(or a commit) to choose the diff base. Pass an optional file or directory path to limit which changed Python production files are mutated.- Read
strength_report.jsonin the target repository root. Treat a killed mutant as behavior caught by the suite and a survived mutant as behavior that the suite did not catch. The runner executes the full test suite for every mutant so indirect, fixture-based, and integration coverage are included.
Analyze survivors
For each item in survived_mutants:
- Inspect its
diffand decide whether it changes observable behavior for valid inputs. - Discard only genuinely equivalent mutants; state why they are equivalent.
- For every real survivor, add one focused test that passes on the original code and fails for the mutant.
Verify that test before suggesting it:
pytest <temporary-test-file> python <skill-path>\scripts\strength.py verify "<mutant-id>" <temporary-test-file>The first command must pass. The verification command must print
VERIFICATION SUCCESSand exit with code 0. Remove the temporary verification file afterward unless the user asks to keep it.
Report results
Use this structure and include only real survivors:
<Total Tests> tests | <Total Mutants> mutants across <Changed Functions Count> changed functions
<Killed Count> killed | <Survived Count> survived | <Equivalent Count> equivalent (discarded)
### SURVIVED
#### <File Path>:<Line Number> <Function Name>
`<Original Expression>` -> `<Mutated Expression>`
Reason for survival: <why the current tests miss this behavior>
Suggested test:
```diff
+ <verified test change>Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

