Skill Doctor 2.0 - Agent-workflow architecture & skill checkup

How the method evolved:

• v1.0 (Matt Pocock, Building Great Agent Skills: The Missing Manual): prompt slimming - who triggers it → steps vs. reference material → Deletion Test.
• v2.0 (practical extension): from "prompt editor" to "agent-workflow architecture reviewer". Adds four defenses: runtime-reality alignment, model/script/runtime ownership, scaffold re-check after model upgrades, and multi-exit transaction recovery.

Language: write the report in the user's language (English in, English out; Chinese in, Chinese out). The table structure stays the same.

🔒 Two consent gates (highest rule - never cross)

1. Gate 1 · Checkup: once the target skill is confirmed, only read, only verify runtime state, only write the report. Never modify a file.
2. Gate 2 · Refactor: after the report, wait for the user to say "refactor / optimize / go ahead" before touching anything. No go-ahead, no edits.

A report is not authorization. Even when the problem is obvious, give the recommendation and wait. A refactor only touches the target skill's directory and its own scripts - never the user's other content or data.

Onboarding (when the user doesn't know which skill to check)

When the user is new, or asks "which skills need a checkup?":

1. Run the triage scan: bash ~/.claude/skills/skill-doctor/scripts/scan-skills.sh [skills_dir ...] (defaults to ~/.claude/skills; project-level skills live in <project>/.claude/skills - pass that too). It prints a table ranked by risk × complexity: lines / split out? / side-effect risk score / days since edit.
2. Pick the 🔴🟠 items - especially short skills with scheduled / outbound / state-writing side effects (hidden high-risk), and skills whose underlying model just changed - then ask the user which ones they actually use most (the filesystem can't show real usage frequency) before recommending.
3. Once they choose, run the checkup below. Always stop at Gate 1; refactoring waits for Gate 2.

Checkup flow (adapts to the skill's type)

Read the target SKILL.md fully first (read long files in segments; include scripts/ and references/ when relevant), then run the gates:

• Pure writing / read-only research skills: focus on gates 1, 2, 3 (re-check taste-type prose and model scaffolding).
• Scheduled / outbound / state-writing / cross-system skills: run all of gates 0-4.

Gate 0: Runtime Reality Check

Principle: a skill's doc states intent; the run ledger is production fact.

• Who really triggers it? Check hooks config, scheduled jobs (cron / launchd / cloud routines / CI), and routing in CLAUDE.md or other entry files.
• Is there two-sources-of-truth drift? Model names, thresholds (e.g. "minimum N sources"), and delivery targets in the skill vs. in the job payload / validation script.
• What do recent runs show? After a recent failure or model upgrade, look at logs / session history for the real sticking point instead of guessing.

Gate 1: Who triggers it (entry design)

• Decide auto-trigger (frequent, low-risk, clear boundary) vs. manual / controlled (rare, high-impact, needs judgment about timing).
• Is the description trigger phrase clear and consistent with the real trigger phrase elsewhere (CLAUDE.md, etc.)? Entry truth converges in one place; strip unrelated instructions leaked in from other systems.
• Remember: every auto-trigger skill's description sits in context permanently - a real token cost.

Gate 2: Structure (steps vs. reference vs. mechanical scripts)

• The main file keeps core decision steps and pointers to branches. Target: readable in ten minutes.
• Templates, long lists, command dictionaries used in only one branch → move to references/, leave one pointer.
• Mechanical operations with fixed input/output (format validation, word counts, fixed-parameter delivery, state updates) → move to scripts/; don't teach the model to hand-type long shell strings.

Gate 3: Three-way test (Deletion / Delegation / Scaffold)

Run every passage through three questions (verdict wording in references/checklist.md):

1. Deletion Test (does it matter?): delete it - does behavior change? If not, it's a no-op / duplicate / sediment: delete or merge.
2. Delegation Test (who owns it?): does this step need open-ended judgment or taste (keep with the model), deterministic execution (give to a script), or lifecycle / fallback reporting (give to the runtime / scheduler)? Never make the model hand-simulate a complex API client (e.g. hand-assembling a JSON argument with thousands of characters of body text).
3. Scaffold Test (generation fit): does this rule bound the final outcome (keep forever: evidence gates, red lines), or is it process scaffolding built for a weaker earlier model (rigid word-count fiddling, step-by-step parameter teaching)? After a model upgrade, keep the outcome gates and remove the process straitjacket.

• Reverse protection: keep leading phrases that anchor a whole way of working (e.g. fail-closed, polished report is not evidence, or Pocock's vertical slice, read first, deletion test).

Gate 4: Transaction & Recovery Test

Only for skills with external side effects (sending messages, writing notes/databases, changing state/git):

• Commit order: strictly generate → validate → deliver → confirm receipt → archive → commit state? Which step is the final commit marker?
• Idempotency: if delivery succeeds but archiving fails, does a persisted receipt stop the rerun from sending twice?
• Honest failure: is "delivery failed but state was updated" ("zombie success") ruled out?

Output: the checkup report

Use the format in references/report-template.md.

Refactor & acceptance (only after Gate 2)

Once the user explicitly agrees:

1. Back up: cp SKILL.md SKILL.md.bak-<date> first (plus any script you'll change).
2. Order of work: pull mechanical steps into deterministic scripts > align with runtime truth > split reference material > remove old-model scaffolding > merge duplicates and delete no-ops.
3. Acceptance matrix (not just line count):
   • compare main-file line count, and grep to confirm key leading phrases and red lines survived;
   • new or changed scripts get bash -n and a --dry-run routing/gate check;
   • record which responsibilities moved from model to script/runtime, and name what to watch on the next real run.