Kotlin Mutation Testing

Audit a Kotlin test class by breaking the production code it exercises, then checking whether the suite of tests in that class catches the break. Mutations that survive the whole class point at production behavior the suite does not constrain.

This is not a full AST-level mutator like Pitest — it's a focused, model-driven audit. You read the production code, pick contextually meaningful mutations, run all tests in the class against each mutation, and report the survivors. The point is to find under-tested production paths, not to grade individual tests.

One mutation is judged by the whole class, not by one test. A mutation is killed if at least one test in the test class fails. A mutation is survived if every test still passes. Never scope the test run to a single method.

Inputs

The user provides one test class, as either:

• A file path (e.g., feature/notes/src/commonTest/kotlin/.../NoteListViewModelTest.kt), or
• A fully qualified class name (e.g., com.plcoding.notes.NoteListViewModelTest).

If the user provides only a class name, resolve the file with find or grep -r "class NoteListViewModelTest" before proceeding.

If the user gives a whole module or directory, stop and ask them to pick a single test class. This skill works one class at a time on purpose — the output must stay reviewable.

Preflight

1. Confirm git is clean for production files

The skill mutates production code and restores it with git checkout --. If the user has uncommitted changes in the files the skill would touch, restoration would destroy their work.

git status --porcelain

If any file under src/main/, src/commonMain/, src/androidMain/, src/jvmMain/, src/iosMain/, or src/desktopMain/ in the module under test is listed as modified, staged, or untracked: stop. Tell the user which files are dirty and ask them to commit or stash before retrying. The test file itself may be dirty — the skill never edits test sources.

2. Detect the module

Walk up from the test file until you hit a build.gradle.kts (or build.gradle). The Gradle module path is the directory path with / replaced by :. Examples:

• feature/notes/build.gradle.kts → :feature:notes
• app/build.gradle.kts → :app

3. Detect the test task

From the same build.gradle.kts:

• If it applies com.android.library or com.android.application → use :module:testDebugUnitTest for unit tests under src/test/ or src/commonTest/.
• Otherwise → use :module:test.
• For KMP modules, prefer :module:jvmTest for JVM-only targets. For commonTest code that runs on multiple targets, pick the JVM variant unless the user says otherwise.
• Instrumented tests (under src/androidTest/, using ComposeTestRule on a device, or annotated with @RunWith(AndroidJUnit4::class) and reading from actual Android framework) require a connected device. If one is not available, skip those test classes with a clear note rather than running.

Workflow

The unit of work is one mutation, not one test. For the chosen test class, enumerate a set of meaningful mutations against the production code those tests exercise, then for each mutation: apply it, run the whole class, classify the result, restore the file, move on.

Restoring the file is mandatory on every exit path, including exceptions, compile errors, and user cancellation.

Step 1 — Map the production surface the class exercises

Read every @Test-annotated method in the class. From all of them collected together, write down:

• Production classes / functions imported from non-test source sets → the candidates to mutate.
• Methods, properties, branches, and return values actually reached by the tests → where mutations should land.
• Assertion shape used by the suite → equality on enums, list equality, assertTrue, verify calls, etc. This shapes which mutations the suite could catch.
• Test type:
  • Unit — only fakes/mocks, pure logic, no framework dependencies, no real DB/HTTP/UI.
  • Integration — real collaborators wired together (in-memory Room, MockWebServer, real repository over a fake data source).
  • E2E / UI — ComposeTestRule, instrumented, drives a real UI flow.

Step 2 — Read the production code

Never pick mutations from the test alone. Open the production file(s) and locate every branch, comparator, return, and side effect the test class touches. Mutations must be chosen against real behavior, not guessed.

Step 3 — Plan the mutations

Build a list of small, behavior-changing edits. Each entry should:

• Touch one line or one token.
• Have a clear behavioral consequence (a correct suite would catch it).
• Cover a different point in the production code — don't pile up redundant variants of the same edit.

Aim for breadth: every guard, every comparator, every return value, every collection operation the suite touches should appear at least once.

A mutation is valid only if a correct suite would definitely catch it. If you can't state in one sentence why the suite should fail, pick a different one.

See the Mutation catalog below for concrete choices.

Step 4 — For each mutation, apply → run class → classify → restore

For every mutation in the list:

1. Apply with Edit — change exactly one line or one token. The diff must be small enough for the user to verify at a glance in the final report.
2. Run the entire test class:

   ./gradlew :module:test --tests "com.example.MyClassTest"

   • For Android modules use :module:testDebugUnitTest instead of :module:test.
   • For KMP modules with a JVM target use :module:jvmTest.
   • Always scope with --tests "ClassName" (no method suffix). Running the whole class is the entire point: the suite's job is to catch the mutation collectively. Filtering down to one method defeats the audit.
3. Classify with the table below.
4. Restore the production file:

   git checkout -- <relative/path/to/ProductionFile.kt>

   Verify restoration worked by re-running git status --porcelain for the touched file — the line must disappear. If it doesn't, stop the audit and tell the user.

Never mutate:

• Test sources (src/test/, src/commonTest/, src/androidTest/, src/jvmTest/, etc.).
• Files outside the module the test belongs to.
• Generated code (anything under build/).

Step 5 — Classify the result

Step 6 — Equivalent mutations

Some mutations don't change observable behavior — for example, password.length < 9 vs. password.length <= 8 are mathematically identical. If a mutation survives, ask: is there any input on which this produces a different program? If no, mark it equivalent, not survived. Equivalent mutations are not coverage gaps; they're impossible to kill by definition.

Step 7 — When to skip mutations

Some production lines aren't usefully mutatable. Skip (don't include) in these cases:

• The line only constructs a data class or returns a constant the test class never reads.
• The branch is behind a DI graph the skill can't resolve from static reading alone (e.g., a platform-specific expect/actual whose actual lives in a sibling module).
• The line calls into a third-party library where mutating the user's code wouldn't change observable behavior.
• Instrumented test class and no connected device → skip the whole audit.

Note any skipped lines in the final report with their reason.

Mutation catalog

Think in terms of what behavior the line encodes, not what pattern matches. Common cases:

Rule of thumb: one line or one token. If your mutation needs more, you're probably mutating the wrong thing — re-read the production code and pick a tighter target.

Final report

After every planned mutation has been processed, print this markdown report to the user:

# Mutation testing report — com.example.MyClassTest

- Mutations applied: 14
- Killed: 11
- Survived: 2
- Equivalent: 1

## Survived mutations (coverage gaps)

### Mutation 1
- **File**: `feature/notes/src/main/kotlin/.../NoteRepository.kt:42`
- **Before**: `if (note.isArchived)`
- **After**: `if (!note.isArchived)`
- **Why the class missed it**: no test in the class exercises the archived branch with an unarchived note; every test passes archived notes only.
- **Suggested test**: a case that constructs an unarchived note and asserts it is not deleted.

### Mutation 2
- **File**: `…`
- **Before**: …
- **After**: …
- **Why the class missed it**: …
- **Suggested test**: …

## Equivalent mutations

### Mutation X
- **File**: `…:N`
- **Before**: `password.length < 9`
- **After**: `password.length <= 8`
- **Justification**: equivalent for all integer inputs; no test could distinguish.

The Why the class missed it line is the most valuable part of the report — it turns a raw "this mutation survived" signal into an actionable suggestion. Write one sentence that names which production behavior is unconstrained, and one sentence that sketches the missing test.

Anti-patterns

• Don't edit test code. The only way to audit the suite fairly is to leave it untouched.
• Don't scope the test run to a single method. Every mutation must be evaluated against the whole test class — a mutation killed by any test is killed.
• Don't mutate more than one place at a time. Compound mutations make the diff unreviewable and confuse classification.
• Don't skip the restore. A crashed audit that leaves the repo mutated is worse than no audit at all. Verify with git status --porcelain between every mutation.
• Don't invent a mutation you can't justify. If you can't explain in one sentence why a correct suite should catch the change, pick a different one or skip the line.
• Don't silently pass when the build fails. A compile error is not a killed mutation; it's a bad mutation.
• Don't conflate survived with equivalent. A survived mutation is a real coverage gap; an equivalent mutation cannot be killed by definition. Treat them separately in the report.