agentleFS
hongyuanchongyuanc/codex-game-studios/.agents/skills/regression-suite/SKILL.md

regression-suite

Skill5 starsChanged 4 months agoMITView on GitHub

Use when bug fixes, critical paths, or release gates need regression coverage mapping and drift detection.

We split this skill into the rules it gives an agent

16 rules. 6 are common, 3 are distinctive, and 7 are very specific to this project.

  • 6

    Common

    Most of the 9 edited versions we read keep these.

  • 3

    Distinctive

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Before default delegation, run python3 ../../../tools/codex_studio/agent_delegation.py resolve --project-root <project-root> --role <role> …

    line 9
  2. Distinctive:

    Ask one decision question per turn and wait for the answer before asking another.

    line 13
  3. Very specific:

    Use request_user_input for structured choices when it is available.

    line 14
  4. Distinctive:

    Use Codex custom agents by role and profile when delegation is useful.

    line 15
  5. Distinctive:

    Treat any approved write as one complete proposed changeset.

    line 16
  6. Common:

    After fixing a bug (confirm a regression test was written or identify gap)

    line 32
  7. Very specific:

    Before a release gate ($gate-check polish requires regression suite exists)

    line 33
  8. Very specific:

    No argument — if a sprint is clearly active (sprint plan exists with in-progress stories), run update.

    line 47
  9. Very specific:

    Read tests/regression-suite.md if it exists.

    line 60
  10. Common:

    If it does not exist: note "No regression suite found — will create one."

    line 65
  11. Common:

    Do not read test file contents unless needed for name-to-test mapping.

    line 77
  12. Common:

    Check for drift indicators:

    line 143
  13. Very specific:

    Before asking, show the exact proposed entries to add, retain, revise, or remove, with evidence paths and reasons.

    line 233
  14. Common:

    After writing (if approved):

    line 239
  15. Common:

    If bug regression gaps > 0: "These bugs can silently return without regression tests.

    line 243
  16. Very specific:

    If coverage drift detected: "Regression suite may be drifting.

    line 245

“Common” is measured: kept by at least 6 in 10 of the 9 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md
<!-- codex-studio-delegation: governed -->

Resolve every role through ../../../.codex/docs/plugin-agent-delegation.md; do not require a repository-local .codex/agents/ or .codex/agent-packs/ tree. Before default delegation, run python3 ../../../tools/codex_studio/agent_delegation.py resolve --project-root <project-root> --role <role> and use only its returned role contract.

Codex Interaction Contract

  • Ask one decision question per turn and wait for the answer before asking another.
  • Use request_user_input for structured choices when it is available; otherwise ask the same concise question directly.
  • Use Codex custom agents by role and profile when delegation is useful.
  • Treat any approved write as one complete proposed changeset. Do not add unlisted files or behavior; pause and request a new approval if scope expands.

Regression Suite

This skill ensures that every bug fix is backed by a test that would have caught the original bug — and that the regression suite stays current as the game evolves. It also detects when new features have been added without corresponding regression coverage.

A regression suite is not a new test category — it is a curated list of tests already in tests/ that collectively cover the game's critical paths and known failure points. This skill maintains that list.

Output: tests/regression-suite.md

When to run:

  • After fixing a bug (confirm a regression test was written or identify gap)
  • Before a release gate ($gate-check polish requires regression suite exists)
  • As part of sprint close to detect coverage drift

1. Parse Arguments

Modes:

  • $regression-suite update — scan new bug fixes this sprint and check for regression test presence; add new tests to the suite manifest
  • $regression-suite audit — full audit of all GDD critical paths vs. existing test coverage; flag paths with no regression test
  • $regression-suite report — read-only status report (no writes); suitable for sprint reviews
  • No argument — if a sprint is clearly active (sprint plan exists with in-progress stories), run update. If ambiguous or no active sprint is detected, use request_user_input:
    • Prompt: "No subcommand specified. Which mode do you want to run?"
    • Options:
      • [A] update — scan new bug fixes this sprint and add missing regression tests
      • [B] audit — full audit of all GDD critical paths vs. existing test coverage
      • [C] report — read-only status report (no writes)

2. Load Context

Step 2a — Load existing regression suite

Read tests/regression-suite.md if it exists. Extract:

  • Total registered regression tests
  • Last updated date
  • Any tests flagged as STALE or QUARANTINED

If it does not exist: note "No regression suite found — will create one."

Step 2b — Load test inventory

file search all test files:

tests/unit/**/*_test.*
tests/integration/**/*_test.*
tests/regression/**/*

For each file, note the system (from directory path) and file name. Do not read test file contents unless needed for name-to-test mapping.

Step 2c — Load GDD critical paths

For audit mode: read design/gdd/systems-index.md to get all systems. For each MVP-tier system, read its GDD and extract:

  • Acceptance Criteria (these define the critical paths)
  • Formulas section (formulas must have regression tests)
  • Edge Cases section (known edge cases should have regression tests)

For update mode: skip full GDD scan. Instead read the current sprint plan and story files to find stories with Status: Complete this sprint.

Step 2d — Load closed bugs

file search production/qa/bugs/*.md and filter for bugs with a Status: Closed or Status: Fixed field. Note:

  • Which story or system the bug was in
  • Whether a regression test was mentioned in the fix description

3. Map Coverage — Critical Paths

For audit mode only:

For each GDD acceptance criterion, determine whether a test exists:

  1. repository search tests/unit/[system]/ and tests/integration/[system]/ for file names and function names related to the criterion's key noun/verb
  2. Assign coverage:
StatusMeaning
COVEREDA test file exists that targets this criterion's logic
PARTIALA test exists but doesn't cover all cases (e.g. happy path only)
MISSINGNo test found for this critical path
EXEMPTVisual/Feel or UI criterion — not automatable by design
  1. Elevate MISSING items that correspond to formulas or state machines to HIGH PRIORITY gap — these are the most likely regression sources.

4. Map Coverage — Fixed Bugs

For each closed bug:

  1. Extract the system slug from the bug's metadata
  2. repository search tests/unit/[system]/ and tests/integration/[system]/ for a test that references the bug ID or the specific failure scenario
  3. Assign:
    • HAS REGRESSION TEST — a test was found that would catch this bug
    • MISSING REGRESSION TEST — bug was fixed but no test guards against recurrence

For MISSING REGRESSION TEST items:

  • Flag them as regression gaps
  • Suggest the test file path: tests/unit/[system]/[bug-slug]_regression_test.[ext]
  • Note: "Without this test, this bug can silently return in a future sprint."

5. Detect Coverage Drift

Coverage drift occurs when the game grows but the regression suite doesn't.

Check for drift indicators:

  • Stories completed this sprint with no corresponding test files in tests/
  • New systems added to systems-index.md since the last regression-suite update
  • GDD sections added or revised since the regression suite was last updated (use repository search on GDD file modification hints if available, or ask the user)
  • tests/regression-suite.md last-updated date vs. current date — if gap > 2 sprints, flag as likely stale

6. Generate Report and Suite Manifest

Report format (in conversation)

## Regression Suite Status

**Mode**: [update | audit | report]
**Existing registered tests**: [N]
**Test files scanned**: [N]

### Critical Path Coverage (audit mode only)
| System | Total ACs | Covered | Partial | Missing | Exempt |
|--------|-----------|---------|---------|---------|--------|
| [name] | [N] | [N] | [N] | [N] | [N] |

**Coverage rate (non-exempt)**: [N]%

### Bug Regression Coverage
| Bug ID | System | Severity | Has Regression Test? |
|--------|--------|----------|----------------------|
| BUG-NNN | [system] | S[N] | YES / NO ⚠ |

**Bugs without regression tests**: [N]

### Coverage Drift Indicators
[List new systems or stories with no test coverage, or "None detected."]

### Recommended New Regression Tests
| Priority | System | Suggested Test File | Covers |
|----------|--------|---------------------|--------|
| HIGH | [system] | `tests/unit/[system]/[slug]_regression_test.[ext]` | BUG-NNN / AC-[N] |
| MEDIUM | [system] | `tests/unit/[system]/[slug]_test.[ext]` | [criterion] |

Suite manifest format (tests/regression-suite.md)

The manifest is a curated index — not the tests themselves, but a registry of which tests should always pass before a release:

# Regression Suite Manifest

> Last Updated: [date]
> Total registered tests: [N]
> Coverage: [N]% of GDD critical paths

## How to run

[Engine-specific command to run all regression tests]

## Registered Regression Tests

### [System Name]

| Test File | Test Function (if known) | Covers | Added |
|-----------|--------------------------|--------|-------|
| `tests/unit/[system]/[file]_test.[ext]` | `test_[scenario]` | AC-N / BUG-NNN | [date] |

## Known Gaps

Tests that should exist but don't yet:

| Priority | System | Suggested Path | Covers | Reason Not Yet Written |
|----------|--------|----------------|--------|------------------------|
| HIGH | [system] | `tests/unit/[system]/[path]` | BUG-NNN | Bug fixed without test |

## Quarantined Tests

Tests that are flaky or disabled (do not run in CI):

| Test File | Function | Reason | Quarantined Since |
|-----------|----------|--------|-------------------|
| (none) | | | |

7. Write Output

Before asking, show the exact proposed entries to add, retain, revise, or remove, with evidence paths and reasons. Then ask for approval of the complete tests/regression-suite.md changeset. Do not update the registry without that approval.

For update mode: append approved new entries; never remove existing entries. For audit mode: rewrite the full manifest with updated coverage data. For report mode: do not write anything.

After writing (if approved):

  • For each HIGH priority gap: "Consider creating the missing regression test before the next sprint. Run $test-helpers to scaffold the test file."
  • If bug regression gaps > 0: "These bugs can silently return without regression tests. The next sprint should include a story to write the missing tests."
  • If coverage drift detected: "Regression suite may be drifting. Consider running $regression-suite audit at the next sprint boundary."

Verdict: COMPLETE — regression suite updated. (If user declined write: Verdict: BLOCKED.)


Collaborative Protocol

  • Never remove existing regression tests from the manifest without explicit user approval — removing a test that was deliberately written is a regression risk itself
  • Gaps are advisory, not blocking — surface them clearly but do not prevent other work from proceeding (except at release gate where regression suite is required)
  • Quarantine is not deletion — tests with intermittent failures should be quarantined (noted in manifest) but not removed; they should be fixed by $test-flakiness
  • Ask before writing — always confirm before creating or updating the manifest

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.