Test Runner Debugger

Guides agents through executing test suites, filtering to isolated failing test cases, diagnosing test assertion errors, and verifying green runs.

1. Purpose

Empower agents to locate, run, and debug automated tests efficiently. An "Assertion Failure" occurs when expected output does not match actual output (expected X, got Y). A "Test Runner" is the framework executing tests (e.g. Jest, Vitest, Pytest, Go test, Cargo test).

2. When to Use

Use this skill when:

• Running tests before and after making code modifications.
• Isolating a single failing test case in a large test suite without running all 500 tests.
• Diagnosing mock failures, unhandled promise rejections, or timeout errors in tests.
• User requests: "Run tests", "Debug failing test", or "Check why test X fails".

3. When Not to Use

• Do not use to write new regression test cases from scratch; use regression-test-writer.
• Do not use to create standalone reproduction scripts; use bug-reproducer.
• Do not use to lint or type-check; use lint-and-typecheck.

4. Required Inputs

• workspace_root: Directory of the project.
• target_test_file: Specific test file or test pattern to run (e.g. tests/test_auth.py or *.test.ts).

5. Missing-Input Behavior

1. If target_test_file is not specified: inspect package.json -> scripts.test, pytest.ini, or Makefile to discover the default test runner.
2. If the entire test suite is slow (> 30s): identify changed files and run only the test files directly covering those files.
3. Never modify test assertions to make a test pass unless the requirement itself has changed.

6. Procedure

Follow these numbered actions:

1. Action 1: Identify Test Framework & Command Pattern
   • Jest / Vitest: npm test -- <path> or npx vitest run <path> -t "<test_name>".
   • Pytest: pytest <path> -k "<test_name>" -v.
   • Go: go test -v -run "^TestName$" ./<pkg>.
   • Cargo (Rust): cargo test <test_name> -- --nocapture.
2. Action 2: Run Targeted Test in Isolation
   • Execute only the failing test file or case first to minimize diagnostic noise.
3. Action 3: Classify Test Failure Mode
   • Assertion Error: Value mismatch (expected 200, got 500).
   • Exception / Crash: Unhandled runtime exception before assertion.
   • Timeout: Asynchronous operation did not complete within threshold (e.g., 5000ms).
   • Mock Misconfiguration: Mock function not called or called with unexpected arguments.
4. Action 4: Trace Root Cause to Source Code
   • Inspect the exact line in source code triggered by the test stack trace.
5. Action 5: Re-Run Targeted Test
   • Confirm targeted test turns green.
6. Action 6: Run Full Test Suite
   • Confirm no collateral test failures were caused.
7. Action 7: Emit Test Execution Report
   • Format per Section 8.

7. Decision Rules

• If test fails due to timeout: check for unclosed database connections, dangling HTTP requests, or missing await.
• If test fails on mock: check whether mock return value matches the expected schema of the real dependency.
• If all tests pass: state count of passed tests and execution duration.

8. Output Format

Emit the test report using this template:

### Test Execution Report

- **Framework**: [e.g., Vitest | Pytest | Go test | Cargo test]
- **Command Executed**: `[Exact CLI command]`
- **Target**: `[Test file or test case name]`
- **Status**: [PASSED | FAILED]
- **Duration**: [e.g., 0.42s]

#### Test Results Summary
- **Tests Run**: [Total]
- **Passed**: [Count]
- **Failed**: [Count]

#### Failure Diagnosis (if any)
- **Failing Test**: `[Test name]`
- **Expected**: `[Expected value]`
- **Actual**: `[Actual value]`
- **Stack Trace Location**: `[file:line]`
- **Prescribed Source Fix**: `[Target source line and fix]`

9. Verification Checklist

• Test was executed using the project's native runner.
• Specific test case was isolated and verified.
• Full test suite was executed to confirm no collateral regressions.
• Assertions were not altered or disabled to force a pass.

10. Common Failures and Recovery

11. Stop or Escalation Conditions

Stop and ask the user when:

• Test failures are caused by missing external infrastructure (e.g., real Redis, PostgreSQL) not present in the local container.

12. Examples

Example 1: Successful Execution

Input:

Project: Node.js / Vitest
Command: npx vitest run tests/auth.test.ts -t "validates token"

Output:

### Test Execution Report

- **Framework**: Vitest
- **Command Executed**: `npx vitest run tests/auth.test.ts -t "validates token"`
- **Target**: `tests/auth.test.ts` ("validates token")
- **Status**: PASSED
- **Duration**: 0.18s

#### Test Results Summary
- **Tests Run**: 1
- **Passed**: 1
- **Failed**: 0

#### Final Verification Status
- Targeted test passed cleanly. Full suite `npm test` verified: 34 passed, 0 failed.

Example 2: Ambiguous or Missing Input

Input:

Run tests.
Context: No test runner specified, tests/ directory has .py and .ts files.

Output:

### Test Execution Report: Ambiguous Test Setup

- **Status**: Halted

The workspace contains both Python and TypeScript files. Please specify:
1. Which test suite you want to run (e.g. Python pytest or Node.js vitest).
2. The specific target test file or feature.

Example 3: Expected Final Output

Output:

### Test Execution Report

- **Framework**: Pytest
- **Command Executed**: `pytest tests/test_orders.py -k test_calculate_tax -v`
- **Target**: `tests/test_orders.py::test_calculate_tax`
- **Status**: PASSED
- **Duration**: 0.08s

#### Test Results Summary
- **Tests Run**: 1
- **Passed**: 1
- **Failed**: 0

#### Final Verification Status
- Verified 1 test passed in 0.08s.