Test Health — Holistic Coverage Measurement

Trigger

• Keywords: test health, coverage measurement, test metrics, coverage trend, test inventory, holistic test audit

When NOT to Use

Workflow

flowchart TD
    U[User: /test-health] --> M{Mode?}
    M --> |quick| Q[Quick Mode]
    M --> |--full| F[Full Mode]

    Q --> Q1[Test Inventory]
    Q1 --> Q2[Consume Coverage Artifacts]
    Q2 --> Q3[Trend Delta]
    Q3 --> QR[Quick Dashboard]

    F --> A[Phase A: /check-coverage]
    A --> B[Phase B: Coverage Collection]
    B --> C[Phase C: /codex-test-review]
    C --> D[Phase D: Aggregate Dashboard]
    D --> T[Trend Snapshot]
    T --> FR[Full Dashboard]

Modes

Quick Mode Workflow

1. Test Inventory: Count test files by layer using Glob (see references/test-count-parsers.md for layer classification). If --scope <path> specified, limit Glob to that directory. If verify-runner cache exists (.claude/cache/verify/), read historical logs for test counts.
2. Coverage Artifacts: Scan for existing coverage artifacts (see references/artifact-formats.md). If --scope specified, scan within scope only. Never execute project commands in quick mode.
3. Trend Delta: Read previous snapshot, compute delta (see references/trend-schema.md). Skip if --no-trend flag is set.
4. Output: Quick Dashboard.

Full Mode Workflow

Phase A: Feature Coverage

Resolve docs path using bash scripts/resolve-feature.sh (same cascade as other skills) — the shim over the wrapper, which emits the full shape with scan_error: true rather than a bare {} however the CLI fails: nonzero exit, signal, partial write, or a payload that is not the agreed shape. It cannot cover node itself being unavailable — the shim would exit 127 with no JSON — so treat an empty or non-JSON reply as a failure too. Gate on scan_error !== false before reading anything else — never on === true, because an empty or non-JSON reply carries no such field at all and the stricter test is false for it. Only once the flag is exactly false does any other field mean what it says: the failure payload sets has_tech_spec false along with everything else, so branching on that field first reports an unreadable corpus as a feature with no documents, and the coverage of a real feature disappears behind a reassuring advisory.

Phase B: Test Inventory + Coverage Collection

1. Count test files by layer (same as quick mode)
2. If --collect flag: execute project coverage command (test:coverage or coverage from package.json)
3. Otherwise: consume existing coverage artifacts (same as quick mode)
4. Parse test runner stdout for test counts (see references/test-count-parsers.md)

Phase C: Qualitative Review

Dispatch /codex-test-review via Skill tool for 5-dimension quality assessment.

Phase D: Aggregate + Trend

1. Aggregate all dimensions into full dashboard
2. Write trend snapshot (see references/trend-schema.md)
3. Output Full Dashboard

Coverage Collection Strategy (Consume-First)

Prohibited: Never auto-install coverage tools (c8, nyc, istanbul, pytest-cov, tarpaulin, jacoco).

Output: Quick Dashboard

## Test Health (Quick)

### Test Inventory
| Layer | Files | Tests | Source |
|-------|-------|-------|--------|
| Unit  | 25    | 47    | cached_stdout |
| Integration | 1 | 12  | cached_stdout |
| E2E   | 0     | —     | file_count |

### Code Coverage
| Metric | Value | Tool | Freshness |
|--------|-------|------|-----------|
| Lines  | 82.3% | c8   | current   |
| Branches | 76.0% | c8 | current   |

### Trend (vs previous)
| Metric | Previous | Current | Delta |
|--------|----------|---------|-------|
| Line coverage | 80.2% | 82.3% | +2.1% |
| Test count | 57 | 59 | +2 |

### Quick Verdicts
| Dimension | Status |
|-----------|--------|
| Has tests for changed files | OK |
| Coverage artifact exists | OK |
| Trend direction | Improving |

Output: Full Dashboard

## Test Health Report (Full)

### Phase A: Feature Coverage
(from /check-coverage): 12/15 documented features have tests (80%)

### Phase B: Code Coverage + Inventory
| Layer | Files | Tests | Passed | Failed | Duration |
|-------|-------|-------|--------|--------|----------|
| Unit  | 25    | 47    | 45     | 2      | 12s      |
| Integration | 1 | 12  | 12     | 0      | 45s      |
| E2E   | 0     | 0     | —      | —      | —        |

| Metric | Value | Source | Tool | Freshness |
|--------|-------|--------|------|-----------|
| Lines  | 82.3% | instrumented_artifact | c8 | current HEAD |
| Branches | 76.0% | instrumented_artifact | c8 | current HEAD |

### Phase C: Quality Findings
(from /codex-test-review):
| Dimension | Rating |
|-----------|--------|
| Happy path | 4/5 |
| Error handling | 3/5 |
| Edge cases | 3/5 |
| Mock quality | 4/5 |

### Phase D: Aggregate Dashboard

#### Trend (vs last 5 runs)
| Run | Date | Line Cov | Tests | Delta |
|-----|------|----------|-------|-------|
| a1b2c3d | 04-01 | 82.3% | 59 | +2.1% / +2 |
| f4e5d6c | 03-31 | 80.2% | 57 | -0.5% / +0 |

#### Verdicts
| Dimension | Status | Detail |
|-----------|--------|--------|
| Test inventory | WARN | No E2E tests |
| Code coverage | OK | 82.3% lines (instrumented) |
| Feature coverage | OK | 80% features covered |
| Quality | WARN | 1 P2 finding |
| Trend | OK | Improving over last 3 runs |
| Changed-file coverage | OK | All changed files have tests |

Anti-Coverage-Theater Guardrails

Gate Policy

v1 implements advisory mode only.

Orchestrator Integration

Cross-Ecosystem Support

Graceful degradation: no artifact + no coverage command = heuristic proxy + source_type: heuristic.

Verification

• Quick mode completes in <15s without executing project commands
• Full mode orchestrates Phase A→B→C→D in sequence
• Coverage artifact consumed correctly (or graceful fallback)
• Trend snapshot written to .claude/cache/test-health/
• Dashboard output includes all dimensions with source transparency

References