agentleFS
Jagoda11Jagoda11/the-jagoda-toolkit/skills/review-ai-compat/SKILL.md

review-ai-compat

Skill0 starsChanged 6 months agoMITView on GitHub

AI-agent compatibility audit. Use when evaluating whether code changes are safely readable, analyzable, and modifiable by AI agents. Checks determinism, explicitness, complexity, and boundary clarity.

The file

Show its rules and text

We split this skill into the rules it gives an agent

21 rules. About 0 are common, 17 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈17

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All changes must optimize for:

    line 13
  2. Distinctive:

    Read the changed files. For each file, run the audit phases below.

    line 37
  3. Very specific:

    Use git commands only — do NOT use gh CLI or GitHub API.

    line 39
  4. Distinctive:

    If any answer is concerning, explain why and what to fix.

    line 55
  5. Very specific:

    Before assessing, gather evidence: Run these checks on the changed files (or all src/ files in baseline mode):

    line 59
  6. Distinctive:

    Verify these rules are respected:

    line 78
  7. Distinctive:

    Every rule violation must cite the exact grep output or file/line that triggered it.

    line 90
  8. Very specific:

    Search for files with similar names or identical content in different locations.

    line 152
  9. Very specific:

    Run: find .claude/skills .agents/skills -name "SKILL.md" -type f and compare.

    line 153
  10. Distinctive:

    If AGENTS.md exists, does it match the name in the AGENTS.md skills table?

    line 171
  11. Distinctive:

    If overall score is below 7, provide:

    line 215
  12. Distinctive:

    No architectural changes required

    line 220
  13. Distinctive:

    No global mutable state

    line 231
  14. Distinctive:

    No business logic in UI rendering

    line 232
  15. Distinctive:

    No DB access outside the dedicated data-access layer

    line 233
  16. Distinctive:

    No undocumented shared abstractions

    line 234
  17. Distinctive:

    No dynamic runtime patching

    line 235
  18. Distinctive:

    No vague "simplify the code" advice.

    line 263
  19. Distinctive:

    Every finding must reference a specific file, function, or pattern.

    line 264
  20. Distinctive:

    If insufficient information to conclude, state: "Insufficient information to conclude".

    line 266
  21. Distinctive:

    After reviewing the output, you may paste the findings into a new prompt:

    line 271

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Review AI-Compat — AI-Agent Compatibility Audit

Purpose

Evaluate whether code changes are safely readable, analyzable, and modifiable by AI agents. This codebase treats AI as an architectural participant — not just a code generator.

All changes must optimize for:

  • Deterministic behavior
  • Explicit contracts
  • Shallow reasoning depth
  • Minimal hidden state
  • Clear module boundaries
  • Structural testability
  • Predictable file organization
  • Minimal cognitive complexity

Instructions

Step 0 — Scope Detection

Detect the repo's default branch:

BASE=$(git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's@^refs/remotes/origin/@@')
[ -z "$BASE" ] && BASE=$(git rev-parse --abbrev-ref HEAD)

Run: git diff $BASE...HEAD --name-only

Read the changed files. For each file, run the audit phases below.

Use git commands only — do NOT use gh CLI or GitHub API.

Phase 1 — AI-Agent Compatibility Check

For each changed file, answer these questions:

#QuestionAnswer
1Does this change introduce hidden state?Yes/No
2Does this increase function complexity or nesting depth?Yes/No
3Does this introduce implicit cross-layer dependencies?Yes/No
4Would an AI agent understand this file in isolation?Yes/No
5Are contracts explicit and typed?Yes/No
6Is side-effect behavior clearly separated?Yes/No
7Are error paths explicit?Yes/No

If any answer is concerning, explain why and what to fix.

Phase 2 — Structural Coding Rules Assessment

Before assessing, gather evidence. Run these checks on the changed files (or all src/ files in baseline mode):

# Find `any` usage (should be zero)
grep -rn ": any\|as any\|<any>" <workspace>/src/ --include="*.ts" --include="*.tsx"

# Find swallowed errors (empty catch blocks)
grep -rn "catch.*{" <workspace>/src/ --include="*.ts" -A2 | grep -B1 "}"

# Find implicit return types on exported functions
grep -rn "^export.*function\|^export const.*=.*=>" <workspace>/src/ --include="*.ts" | grep -v ":"

# Find global mutable state (let/var at module level)
grep -rn "^let \|^var " <workspace>/src/ --include="*.ts" --include="*.tsx"

# Measure file lengths (flag files over 200 lines)
wc -l <workspace>/src/**/*.ts 2>/dev/null | sort -rn | head -20

Verify these rules are respected:

RuleCheck
Pure-First PolicyAre functions pure where possible? Side effects at edges only?
Shallow ModulesNo deep inheritance or decorator chains?
No Implicit SingletonsAll dependencies injected?
Typed BoundariesNo any? No implicit return types?
Explicit Error SurfacesNo swallowed errors? No silent fallback?
File CohesionOne responsibility per file?
Predictable LayoutStandard directory structure? No magic dynamic resolution?

Every rule violation must cite the exact grep output or file/line that triggered it.

Phase 2a — Named Anti-Pattern Detection

Detect these labeled failure modes in the diff. Flag with the name, cite the line.

LabelDetection
drive_by_refactoringDiff contains edits outside the scope of the reported fix/feature
style_driftQuote style, type annotations, or whitespace changed in the same diff as a functional change
speculative_featuresNew options, configs, abstractions not in the user ask
hidden_assumptionsCode commits to a scope/format/fields without asking (missing clarification turn)

Format each finding as: [label] <file>:<line> — <specific issue>

Phase 3 — AI-Agent Readability Audit

Assess each changed file across these dimensions:

DimensionWhat to evaluate
DeterminismGiven the same input, does the code always produce the same output?
Contract explicitnessAre all inputs, outputs, and side effects typed and documented?
Hidden stateIs there global, mutable, or closure-captured state?
Call graph depthHow many files must an agent read to understand this one?
Mutation boundariesIs it clear where state changes happen?
Dependency injectionAre external dependencies visible and injectable?
Error explicitnessAre all error paths handled and surfaced?
Test determinismCan tests run in any order without shared state?

Measure call graph depth and import density for changed files:

# Count imports per file (high count = high coupling)
for file in <changed-files>; do
  echo "$(grep -c "^import " "$file") imports — $file"
done | sort -rn

# Find files that import from 5+ different modules
grep -c "^import " <workspace>/src/**/*.ts 2>/dev/null | awk -F: '$2 >= 5 {print}'

Then identify:

  • Parts likely to cause incorrect automated refactors — Why?
  • Parts likely to cause patch conflicts — Why?
  • Parts that rely on implicit architectural knowledge — What knowledge?

Phase 4 — Structural Integrity Checks

These checks are MANDATORY. Do not skip any. Run each one and report the result.

Detect dual-tooling layout first:

DUAL_TOOLING=0
[ -d .agents/skills ] || [ -f AGENTS.md ] && DUAL_TOOLING=1

Phases 4a, 4b, 4c only run when DUAL_TOOLING=1. Otherwise skip them and note "single-tooling repo — dual-tooling checks skipped".

4a — Duplicate detection (dual-tooling only): Search for files with similar names or identical content in different locations. Run: find .claude/skills .agents/skills -name "SKILL.md" -type f and compare. Are there orphaned, renamed, or duplicate files? List them.

4b — Cross-reference validation (dual-tooling only): For every skill name referenced in any file (router tables, AGENTS.md, CLAUDE.md):

  • Does the actual skill file exist in .claude/skills/ AND .agents/skills/?
  • List any referenced skills that do not exist.

4c — Symmetry check (dual-tooling only): For each skill, compare .claude/skills/<name>/SKILL.md with .agents/skills/<name>/SKILL.md.

  • Content must be identical except for allowed-tools (Claude Code only).
  • Report any content differences beyond allowed-tools.

4d — Naming consistency:

  • Does the directory name match the name: field in YAML front matter?
  • If AGENTS.md exists, does it match the name in the AGENTS.md skills table?
  • Are there any stale references to old names?

4e — Implicit knowledge audit: List every piece of information an agent would need to work with these files that is NOT written in the files themselves. Examples:

  • Which branch is the base branch?
  • Which directory is for which tool?
  • Which names are canonical vs deprecated?

Phase 5 — AI Patch Safety Score (MANDATORY)

The patch safety score is MANDATORY — without a numeric score, findings stay vague and there's no way to track whether changes are making the codebase more or less agent-friendly over time. Report all scores. Every dimension gets a number, even if it's a 10.

Score the change 1-10 for AI-modifiability. Do not skip any dimension:

DimensionScore (1-10)Justification
Local understandabilityCan an agent reason about this file without reading 5 others?
State explicitnessIs all state visible and traceable?
Control flow simplicityCan the execution path be followed linearly?
Refactor safetyWhat would break during automated refactor?
Tribal knowledge dependencyDoes understanding require unwritten context?

Scoring action table:

ScoreAction
9-10Report — safe for agents, no action needed
7-8Report — acceptable, minor improvements optional
4-6Report — flag for review, improvements recommended
1-3Report — redesign required, not safe for agent modification

Calculate an overall average score.

Verdict rules:

  • PASS — average ≥ 7
  • FLAG — average 4–6
  • REDESIGN — average < 4

Phase 6 — Improvements

If overall score is below 7, provide:

Minimal Improvements (low disruption):

  • Specific changes to improve agent readability
  • No architectural changes required

Structural Improvements (if score below 4):

  • Architectural changes to restore agent compatibility
  • File splits, dependency extraction, state isolation

Anti-Entropy Rules

Flag violations of:

  • No global mutable state
  • No business logic in UI rendering
  • No DB access outside the dedicated data-access layer
  • No undocumented shared abstractions
  • No dynamic runtime patching
  • No circular dependencies

Contract

Append this JSON block to every audit output — it is the verifiable contract:

{
  "agent": "review-ai-compat",
  "branch": "<branch>",
  "date": "<today>",
  "verdict": "PASS|FLAG|REDESIGN",
  "dimensions": {
    "localUnderstandability": 0,
    "stateExplicitness": 0,
    "controlFlowSimplicity": 0,
    "refactorSafety": 0,
    "tribalKnowledgeDependency": 0
  },
  "averageScore": 0,
  "findings": ["specific issues"],
  "improvements": ["specific recommendations"]
}

Output Constraints

  • No vague "simplify the code" advice.
  • Every finding must reference a specific file, function, or pattern.
  • Separate facts from assumptions.
  • If insufficient information to conclude, state: "Insufficient information to conclude".
  • This audit measures AI compatibility — not general code quality.

Optional: Self-Correction (Manual)

After reviewing the output, you may paste the findings into a new prompt:

"Here are the findings from my AI-compat audit. Which of these might be incorrect due to missing context? What additional data would increase confidence?"

IMPORTANT: This step must be human-initiated — never auto-dismiss findings. The human decides what to act on.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.