agentleFS
me2reshme2resh/apexyard/.claude/skills/mutation-test/SKILL.md

mutation-test

Skill542 starsChanged 11 days agoMITView on GitHub

Mutation-testing sensor — language-dispatched (Stryker/MutPy/go-mutesting/mutant), milestone-cadence not per-PR, exit-3 graceful-degrade.

  • Installs packages

We split this skill into the rules it gives an agent

7 rules. 0 are common, 1 is distinctive, and 6 are very specific to this project.

  • 0

    Common

    Most of the 7 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If invoked without an argument, use $PWD.

    line 79
  2. Very specific:

    If $PROJECT_PATH is under <workspace_dir>/<name>/...

    line 83
  3. Very specific:

    Read mutation.runner from .claude/project-config.json (merge over .claude/project-config.defaults.json):

    line 104
  4. Distinctive:

    Before running anything, verify the chosen runner is installed:

    line 125
  5. Very specific:

    If RUNNER_MISSING=1, print the install advisory and exit 3:

    line 136
  6. Very specific:

    Write to <projects_dir>/<name>/quality/mutation-<YYYY-MM-DD>.md.

    line 194
  7. Very specific:

    If yes → run /task with the prefilled body (skill stays out of gh issue create directly.

    line 287

“Common” is measured: kept by at least 6 in 10 of the 7 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Writing: read docs/skills/shared/writing-rule.md before you write a durable artifact.

/mutation-test — Behaviour-Quality Sensor

Runs mutation testing against a project to measure whether the test suite constrains behaviour, not just executes lines. Coverage % answers "did the test run this line?"; mutation testing answers "if I broke this line, would the test catch it?".

Pairs with /launch-check (fans out to this skill at milestone boundaries) and complements the existing > 80% coverage gate from .claude/rules/workflow-gates.md. Not run per-PR — see § "When to use this" for the cadence rationale.

Runtime requirements

DependencyUsed forWithout it
bash ≥ 4The skill itselfRequired
gitProject detection + path resolutionRequired
stryker (@stryker-mutator/core)TS / JS mutation runnerSkill exits 3 with the install one-liner if TS/JS detected
mut.py (MutPy)Python mutation runnerSame shape
go-mutestingGo mutation runnerSame shape
mutant (mutant-rspec / mutant-minitest)Ruby mutation runnerSame shape
jqStryker JSON report parsingRequired when Stryker is the chosen runner

Same disclosure shape as /pdf and /process — disclosed up front, surfaced when invoked, never silently fails.

Path resolution

source "$(git rev-parse --show-toplevel)/.claude/hooks/_lib-read-config.sh"
source "$(git rev-parse --show-toplevel)/.claude/hooks/_lib-portfolio-paths.sh"
projects_dir=$(portfolio_projects_dir)
workspace_dir=$(portfolio_workspace_dir)

Defaults to single-fork mode. Split-portfolio v2 adopters resolve to the sibling private repo transparently. Don't hardcode projects/ or workspace/ literals.

Usage

/mutation-test                          # run against cwd (must be inside workspace/<name>/)
/mutation-test workspace/example-app    # run against an explicit project path
/mutation-test --language=python        # force the Python runner (skip detection)
/mutation-test --runner=stryker         # force a specific runner regardless of language
/mutation-test --threshold=70           # one-off threshold override (doesn't write to config)
/mutation-test --check-only             # report which runners are installed, do nothing else

When to use this

TriggerUse /mutation-test?
Milestone boundary (epic done, release prep, launch check)Yes — /launch-check dispatches automatically
Quarterly health checkYes — explicit invocation
Weekly cron via CI workflowYes — adopter wires a scheduled workflow
Per-PR pre-merge gateNo — too slow (20–40 min on a medium codebase). Rex + the coverage gate cover per-PR test quality.
Pre-deploy smokeNo — use /launch-check for a full sweep instead
After adding new test infrastructureYes — confirm the new tests actually constrain
First-time on a brownfield projectYes — expect a low score on the first run; treat it as a baseline

The skill is NOT wired to any pre-commit, pre-push, or merge-gate hook. It runs only when explicitly invoked or when /launch-check fans out to it.

Process

Step 1 — Resolve the project

PROJECT_PATH="${1:-$PWD}"
if [ ! -d "$PROJECT_PATH" ]; then
  echo "/mutation-test: project path not found: $PROJECT_PATH" >&2
  exit 2
fi
cd "$PROJECT_PATH"

If invoked without an argument, use $PWD. If $PWD is the ops-fork root (contains onboarding.yaml or .apexyard-fork), refuse — the audit is for managed projects, not the framework repo itself.

Resolve <project-name> from the path:

  1. If $PROJECT_PATH is under <workspace_dir>/<name>/... → name is that path segment.
  2. Otherwise fall back to basename "$PROJECT_PATH" and warn the operator to /handover for cross-machine trend continuity.

Step 2 — Detect the language

File-count heuristic across the project tree (excluding node_modules, .venv, vendor, dist, build, .next):

ts_count=$(find . -type f \( -name '*.ts' -o -name '*.tsx' \) ! -path '*/node_modules/*' ! -path '*/dist/*' | wc -l)
js_count=$(find . -type f \( -name '*.js' -o -name '*.jsx' -o -name '*.mjs' \) ! -path '*/node_modules/*' ! -path '*/dist/*' | wc -l)
py_count=$(find . -type f -name '*.py' ! -path '*/.venv/*' ! -path '*/__pycache__/*' | wc -l)
go_count=$(find . -type f -name '*.go' ! -path '*/vendor/*' | wc -l)
rb_count=$(find . -type f -name '*.rb' ! -path '*/vendor/*' | wc -l)

The language with the highest count wins. TS and JS collapse into the same dispatch ("ts/js"). Ties: prefer the language with the matching config-block runner if set; otherwise prefer ts/js > python > go > ruby (alphabetical-of-the-popular-stack tiebreak).

--language=<lang> overrides detection. Mixed-language projects audit only the dominant language in a single run; the report flags the other languages as "not audited in this run; re-run with --language=<lang> to cover them".

Step 3 — Pick the runner

Read mutation.runner from .claude/project-config.json (merge over .claude/project-config.defaults.json):

{
  "mutation": {
    "runner": {
      "ts": "stryker",
      "js": "stryker",
      "python": "mutpy",
      "go": "go-mutesting",
      "ruby": "mutant"
    },
    "threshold": 60
  }
}

The configured runner wins. --runner=<name> flag wins over config. Otherwise use the default for the detected language.

Step 4 — Graceful-degradation check

Before running anything, verify the chosen runner is installed:

case "$RUNNER" in
  stryker)       command -v stryker      >/dev/null 2>&1 || RUNNER_MISSING=1 ;;
  mutpy)         command -v mut.py       >/dev/null 2>&1 || RUNNER_MISSING=1 ;;
  go-mutesting)  command -v go-mutesting >/dev/null 2>&1 || RUNNER_MISSING=1 ;;
  mutant)        command -v mutant       >/dev/null 2>&1 || RUNNER_MISSING=1 ;;
esac

If RUNNER_MISSING=1, print the install advisory and exit 3:

✗ No mutation tester installed for <language> (chosen runner: <RUNNER>).

Per-language install one-liners:
  TS / JS  — npm install --save-dev @stryker-mutator/core
              (then add stryker.conf.json — see https://stryker-mutator.io/docs/stryker-js/)
  Python   — pip install mutpy
              (https://github.com/mutpy/mutpy)
  Go       — go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest
              (https://github.com/zimmski/go-mutesting)
  Ruby     — gem install mutant-rspec   (or mutant-minitest)
              (https://github.com/mbj/mutant)

Install the runner for your project's language and re-run /mutation-test.
This skill never bundles a mutation runner — same graceful-degrade shape
as /pdf and /process.

Exit code 3. Same shape as /pdf and /process. The rest of the framework is unaffected.

--check-only reports which runners are installed without invoking any and exits 0 if at least one is, exit 3 if none.

Step 5 — Invoke the runner

The dispatch table (skill resolves the per-runner invocation):

RunnerInvocationReport shape consumed
strykernpx stryker run --reporters json,clear-text then jq the reports/mutation/mutation.jsonJSON: files.<path>.mutants[] with status ∈ {Killed, Survived, Timeout, NoCoverage, CompileError, RuntimeError}
mutpymut.py --target <module> --unit-test <tests> --report-html out/ (or --report yaml)Parse the YAML/HTML survivors list
go-mutestinggo-mutesting ./...Parse the trailing PASS/FAIL counts + per-mutant lines
mutantmutant run (project's .mutant.yml config)Parse the summary table

The runner's stdout/stderr stream through (so operators can watch progress on long runs). The skill collects the report file at the end.

Timeout cap: 90 minutes wall-clock. Beyond that, the skill kills the runner and emits an incomplete report flagging the time-out. (Configurable via mutation.timeout_minutes.)

Step 6 — Parse results

Normalise the runner's output into a common record:

total_mutants    = sum of all mutant outcomes
killed           = mutants the tests caught
survived         = mutants the tests did NOT catch
timed_out        = runner timed out before the test could finish
no_coverage      = mutated line not exercised by any test
compile_error    = mutant didn't compile (counts as killed in most conventions)
runtime_error    = runner crashed on this mutant (skip)
score_pct        = round(100 * killed / (total_mutants - no_coverage - runtime_error))

The denominator deliberately excludes no_coverage and runtime_error — the convention is to score only the mutants that were genuinely tested. compile_error counts as killed (Stryker's default; MutPy and go-mutesting align).

Step 7 — Render the report

Write to <projects_dir>/<name>/quality/mutation-<YYYY-MM-DD>.md. If a report for today already exists, append -NN (start at -01) so re-runs don't clobber.

Report shape (six sections):

# Mutation report — <project> — <YYYY-MM-DD>

| Field | Value |
|-------|-------|
| Project   | <name> |
| Language  | <ts|js|python|go|ruby> |
| Runner    | <stryker|mutpy|go-mutesting|mutant> |
| Threshold | <N>% |
| Command   | `<exact invocation>` |
| Duration  | <HH:MM:SS> |

## Score

**<killed> / <denominator> = <score_pct>%** — <PASS|WARN below threshold of <N>%>

## Summary

| Outcome | Count | Notes |
|---------|-------|-------|
| Killed         | <N> | Test suite caught the mutation |
| Survived       | <N> | **Test gap** — investigate top-5 below |
| Timed out      | <N> | Counts as killed in most conventions |
| No coverage    | <N> | Mutated line not exercised by any test |
| Compile error  | <N> | Counts as killed |
| Runtime error  | <N> | Excluded from score |

## Top-5 survived mutants

For each (up to five), with file:line, mutator name, original snippet, mutated snippet, why-it-survived hint:

### 1. `src/foo.ts:42` — ArithmeticOperator (`+` → `-`)

```ts
// Original
return a + b;

// Mutated
return a - b;
```

**Hint:** the test asserts `result` is a number, not the specific value. Tighten the assertion to `expect(result).toBe(5)`.

### 2. ...

(Up to 5 — if there are more, summarise the long tail as a count by mutator type.)

## Trend (last 5 runs)

| Date | Score | Threshold | Verdict |
|------|-------|-----------|---------|
| 2026-05-20 | 64% | 60% | PASS |
| 2026-04-29 | 58% | 60% | WARN |
| ...        | ...  | ...   | ...    |

(Section omitted if there are no prior reports.)

## Recommendations

- File-specific test gaps surfaced above
- Equivalent-mutant suppression candidates (mutants the runner classified survived but that look semantically equivalent)
- Runner-config tweaks (e.g. `stryker.conf.json` `ignorePatterns`)

Step 8 — Emit a one-line verdict to stdout

Same scannable shape as /launch-check:

✓ MUTATION TEST — <project> — <score_pct>% (threshold <N>%) — PASS
  Report: projects/<name>/quality/mutation-<YYYY-MM-DD>.md

or:

⚠ MUTATION TEST — <project> — <score_pct>% (threshold <N>%) — WARN
  Below threshold. Top-5 survived mutants in:
  projects/<name>/quality/mutation-<YYYY-MM-DD>.md

WARN does not block anything mechanically. The signal is advisory.

Step 9 — Optional auto-ticket offer

Want me to file a [Task] ticket to address the top survived mutants? [y/N]

If yes → run /task with the prefilled body (skill stays out of gh issue create directly; the structured-ticket gate fires through /task). If no → exit cleanly.

Config

.claude/project-config.defaults.json ships a mutation block:

"mutation": {
  "runner": {
    "ts": "stryker",
    "js": "stryker",
    "python": "mutpy",
    "go": "go-mutesting",
    "ruby": "mutant"
  },
  "threshold": 60,
  "timeout_minutes": 90
}

Adopters override per-project in .claude/project-config.json:

{
  "mutation": {
    "threshold": 70,
    "runner": { "python": "cosmic-ray" }
  }
}

(Per-key shallow-merge — overriding runner.python doesn't blow away the other languages' defaults.)

Rules

  1. Never run per-PR. The skill is milestone-boundary + on-demand + weekly-cron only. The framework does not wire it to any pre-commit, pre-push, or merge-gate hook.
  2. Graceful-degrade on missing runner. Exit 3 + advisory — never block adopters who don't want the audit. Same shape as /pdf and /process.
  3. Report is dated + per-project. projects/<name>/quality/mutation-<YYYY-MM-DD>.md. Re-runs on the same day append -NN rather than clobber.
  4. Below-threshold is WARN, not FAIL. Mutation testing is a leading indicator, not a launch blocker. The verdict surfaces the gap; the operator decides whether to address now or in the next sprint.
  5. Run from inside the project workspace. The skill checks workspace/<name>/, not the ops repo.
  6. No ticket creation without explicit operator yes. Step 9 offers; doesn't auto-file.
  7. Mixed-language projects audit only the dominant language per run. Re-run with --language=<other> to cover the rest.

Implementation notes

FilePurpose
.claude/skills/mutation-test/SKILL.mdThis file — the skill spec
.claude/skills/mutation-test/detect.shLanguage detection + runner-availability check (shared between full and --check-only modes)
.claude/skills/mutation-test/tests/smoke.shLanguage-detection + graceful-degradation + report-shape tests
.claude/project-config.defaults.json → mutation.*Default runner map + threshold + timeout
.claude/skills/launch-check/SKILL.md § "10. Behaviour quality"The launch-check dimension that dispatches here

Design rationale: see docs/agdr/AgDR-0045-mutation-test-skill.md.

See also

  • AgDR-0045 — runner choices, threshold rationale, why-not-per-PR, graceful-degrade pattern, report location convention
  • .claude/skills/launch-check/SKILL.md — milestone-boundary umbrella that fans out to this skill
  • .claude/skills/pdf/SKILL.md — the graceful-degrade shape this skill mirrors
  • .claude/skills/process/SKILL.md — sibling skill with the same exit-3 install-advisory pattern
  • .claude/rules/workflow-gates.md — the existing > 80% coverage gate that this skill complements (coverage = was-it-executed; mutation = does-it-constrain)

Part of ApexYard — multi-project SDLC framework for Claude Code · MIT.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.