agentleFS
tamldtamld/defense-in-depth/.agents/skills/skill-impact-predictor/SKILL.md

skill-impact-predictor

Skill4 starsChanged 7 months agoMITView on GitHub

Trace import graph + test coverage of files about to be modified and produce an impact report before any code changes are made.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 5 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Philosophy: Every change to a Guard or core module has a blast radius.

    line 15
  2. Distinctive:

    Before any implementation begins on a DiD change, trace the dependency graph of the files about to be modified and produce a structured …

    line 21
  3. Distinctive:

    Read the issue / PR description.

    line 49
  4. Distinctive:

    All HIGH rules are listed before any MEDIUM rule so that "first match wins" cannot let a lower severity shadow a non-negotiable HIGH.

    line 102
  5. Distinctive:

    Write the report to the location agreed with the human (typically a comment on the issue or a scratch file the agent links from the PR).

    line 108

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

SKILL: skill-impact-predictor

Identity

Role: The Blast Radius Calculator Philosophy: Every change to a Guard or core module has a blast radius. Mapping it before editing prevents the silent regressions that this project's pre-commit pipeline cannot catch.


Mission

Before any implementation begins on a DiD change, trace the dependency graph of the files about to be modified and produce a structured impact report that captures:

  1. Direct targets — the files the agent intends to edit.
  2. Inbound consumers — every file that imports or re-exports a direct target.
  3. Test coverage — which tests under tests/ exercise each direct target.
  4. Risk level — LOW / MEDIUM / HIGH per the rules in §Workflow Step 5.
  5. Recommendation — proceed, split the work, or write tests first.

The skill is a planning artifact. It runs before the diff exists. Running it after coding is theater and does not satisfy the hard gates below.


Hard Gates

IDGateIf FAIL
G1Run BEFORE any source edit. The agent's working tree must be clean (or contain only doc edits) when the report is produced.STOP — post-hoc analysis is rejected.
G2The report cites real files in src/, tests/, or .agents/. No invented paths.STOP — re-run discovery with grep/rg.
G3If a direct target is a public-API file AND risk is HIGH AND test coverage of the target is below 70% by file count, BLOCK and demand tests first. The public-API set is the union of (a) the library entry point src/index.ts and every file it re-exports, including transitive barrels such as src/guards/index.ts and src/federation/index.ts, and (b) the CLI entry point src/cli/index.ts plus any file imported directly by it (CLI surface is not re-exported by the library, but is a public surface for end users via npx defense-in-depth …). The agent MUST resolve set (a) by reading the current src/index.ts at analysis time — never from a cached list.BLOCK — return to issue queue.
G4If blast radius (direct + inbound consumers) exceeds 50 files, HALT and require splitting the work into smaller issues.HALT — open child issues, do not proceed in one PR.
G5The skill must NOT widen its scope into refactoring, fixing unrelated issues, or modifying tests of unrelated modules. Pure analysis only.STOP — file separate issues for collateral findings.

Workflow

Step 1 — Define the explicit target list

Read the issue / PR description. Extract every file path the agent intends to edit. If the description says only "fix the federation guard", expand to a concrete list (e.g. src/guards/federation.ts, src/federation/file-provider.ts) and confirm with the human before continuing.

Step 2 — Trace inbound consumers (1st degree)

For each direct target, find every file that imports it. Preferred tool: if the CRG MCP server is mounted and exposes get_impact_radius_tool, use it — it understands the TypeScript module graph and avoids the blind spots listed below. Fall back to rg only when CRG is unavailable.

# rg fallback — run from repo root
rg -l "from ['\"][^'\"]*<basename>['\"]" --type ts src tests
rg -l "require\\(['\"][^'\"]*<basename>['\"]\\)" src tests

Known blind spots of the rg regex above (document them in the Impact Report when relevant):

  1. Dynamic imports — await import("...") and import(condition ? a : b) are NOT matched. If the codebase uses dynamic imports for the target's basename, manually grep for import\( and inspect.
  2. Barrel re-exports — a file that re-exports the target via export * from "./<basename>" will be matched, but consumers of THAT barrel won't be — Step 3 (2nd degree) is what catches them. Do not stop at depth 1 if a 1st-degree match is itself a barrel.
  3. .js extension imports — TypeScript NodeNext / ESM style from "./<basename>.js" IS matched by the regex (the basename appears verbatim), but if the target is renamed across an .ts ↔ .js boundary, audit manually.

Record the absolute count and the file list. Pay special attention to:

  • src/guards/index.ts (barrel export — touching it ripples to every guard consumer)
  • src/core/types.ts (Guard interface — touching it ripples to every guard)
  • src/core/engine.ts (pipeline runner — touching it ripples to every CLI command)
  • src/cli/index.ts (CLI router — touching it ripples to all did * subcommands)

Step 3 — Trace 2nd-degree consumers

For every 1st-degree consumer, repeat Step 2. Stop at depth 2 unless the human asks for deeper. Cap the union of 1st + 2nd degree at 50 files (G4).

Step 4 — Map test coverage

For each direct target, identify test files that exercise it.

# Strict: tests that import the target by relative path
rg -l "from ['\"]\\.\\./.*<basename>['\"]" tests
# Loose: tests that mention the basename anywhere
rg -l "<basename>" tests

Record both counts. A direct target with 0 strict-import tests is a coverage gap even if loose mentions exist.

Step 5 — Score risk

Apply the rules in order; first match wins.

RuleRisk
Direct target mutates the Finding.severity contract or registry shapeHIGH
Direct target is a public-API file AND inbound consumers > 5HIGH
Direct target is a Guard (src/guards/*.ts) with 0 strict-import testsHIGH
Direct target modifies an exported type/interface in src/core/types.tsHIGH
Inbound consumers (1st + 2nd degree) > 20MEDIUM
All other casesLOW

All HIGH rules are listed before any MEDIUM rule so that "first match wins" cannot let a lower severity shadow a non-negotiable HIGH. New HIGH conditions added in future revisions MUST stay above the MEDIUM row.

The Finding.severity rule is non-negotiable per SKILL_STANDARDS.md §DiD-Specific Constraints — severity changes propagate to every CI gate consumer and to the federation child report shape.

Step 6 — Emit the report

Write the report to the location agreed with the human (typically a comment on the issue or a scratch file the agent links from the PR). Use this exact structure:

# Impact Report — <issue or PR id>

## Direct targets
- src/...
- src/...

## Inbound consumers (1st degree, count = N)
- src/...
- src/...

## Inbound consumers (2nd degree, count = M)
- src/...

## Test coverage (strict imports)
| Target | Strict-import tests | Loose mentions |
|:--|:-:|:-:|
| src/... | tests/... | ... |

## Risk level
HIGH | MEDIUM | LOW — <one-sentence justification>

## Recommendation
proceed | split (open child issues #X, #Y) | write tests first

Output Contract

ItemRequirement
Working tree state when the report is producedClean of source edits (G1).
File paths in the reportAll cited files exist on disk in this repo (G2).
Risk levelOne of HIGH / MEDIUM / LOW with a one-sentence justification grounded in §Workflow Step 5.
RecommendationAligned with the gates: HIGH + low coverage → "write tests first"; > 50 files → "split"; otherwise "proceed".
Evidence tag for each citation[CODE] when the file exists on disk. Never [HYPO].

Anti-Patterns

  1. Post-hoc impact reports. Running the skill after the diff is already written produces a description of what changed, not a prediction of what could break. The whole value is pre-implementation.
  2. Stopping at direct imports. Touching src/core/types.ts looks like a 3-file change until you trace the Guard interface ripple — every file in src/guards/ becomes 1st-degree, every test in tests/ becomes 2nd-degree. Always trace depth 2 for type/interface changes.
  3. Treating loose rg mentions as test coverage. A test file that mentions "federation" in a string literal is not coverage. Coverage requires a strict import of the target module under test.
  4. Splitting the report across the PR description and the commit message. The report is a single artifact. If it does not fit the PR description, link to a scratch file in the PR comments.
  5. Allowing the skill to bleed into refactoring. The moment the agent edits a file outside the discovery commands above, the report is invalidated by G1. Refactor work belongs in skill-surgical-refactorer after the report is approved.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.