Skill Improver

Purpose

Read the feedback log, identify patterns, run eval checks, propose a targeted patch per skill, apply it after user confirmation, and commit to git. Works for any domain skill without configuration.

Phase 1 — Read and parse feedback

1. Read ~/.claude/skill-data/feedback-log.md.
2. Extract all entries with status: open.
3. If there are zero open entries, stop and say: "No open feedback entries found. Use $skill-feedback to log issues first."
4. Group entries by skill, then by section within each skill.

5. Print a summary table:

   Skill	Section	Open entries	Max severity
   ...	...	...	...

6. Mark a (skill, section) pair as actionable if it has 2 or more open entries, OR if any single entry has severity 4 or 5. Only actionable pairs will be patched. Skip the rest silently.

Phase 2 — Read the target skill

For each skill with at least one actionable pair:

1. Confirm the folder ~/.claude/skills/<skill>/ exists. If it does not, skip this skill and note: "Skill folder not found for <skill> — skipping."
2. Read ~/.claude/skills/<skill>/SKILL.md.
3. Identify the text block that corresponds to the flagged section.
4. Note the pattern across feedback entries (e.g. "two entries say the same section produced output the user could not use directly").

Phase 3 — Run eval checks

For each skill being improved:

1. Check if ~/.claude/skill-data/eval-prompts/<skill>.prompts.csv exists. If it does not, skip this phase for that skill and note: "No eval prompts for <skill> — populate ~/.claude/skill-data/eval-prompts/<skill>.prompts.csv to enable this phase."

2. Read the CSV. Expected columns (in order): id, prompt, asset, should_trigger, must_contain, must_not_contain

   • asset: relative path to an image file under ~/.claude/skill-data/eval-prompts/assets/, or empty string if no image.
   • should_trigger: true or false
   • must_contain: comma-separated strings that must appear in the output when the skill triggers. Use empty string for no check.
   • must_not_contain: comma-separated strings that must NOT appear. Use empty string for no check.
3. For each row, reason against the current SKILL.md:
   • Would this prompt trigger the skill based on its name and description?
   • If triggered, would the output satisfy must_contain given the current instructions?
   • Would must_not_contain strings be absent?
   • If the row has a non-empty asset field, load the image at that path and include it alongside the prompt when reasoning about whether the skill would produce the correct output. If the file does not exist at that path, mark the row as SKIP and note: "asset file not found: <path>"

4. Report results before the patch:

   Eval — <skill> (before patch):

   id	prompt (truncated)	expected trigger	result	reason
   ...	...	...	PASS/FAIL	...

Phase 4 — Propose patch

For each actionable (skill, section) pair (highest severity first, one per run):

1. Write the proposed change as an explicit before/after diff:

   FILE: ~/.claude/skills/<skill>/SKILL.md
   SECTION: <section>
   
   BEFORE:
   <exact current text to be replaced>
   
   AFTER:
   <replacement text>
   
   Reason: <one sentence linking the change to the feedback pattern>

2. Rules for writing patches:
   • Replace only the flagged section. Do not touch any other part of the file.
   • Prefer adding a concrete example over rewriting an abstract rule.
   • Reinforce a changed rule with a > 💡 Lesson: bullet directly beneath it, quoting the pattern observed in the feedback.
   • Never remove quality gates, required sections, or output structure.
   • Keep the change minimal. One fix per actionable pair.
   • If multiple sections are actionable for the same skill, fix the highest severity one only. The others stay open for the next run.

3. Show the diff and ask: "Apply this patch? Reply yes / no / edit.

   • yes → apply and commit
   • no → skip, entries remain open
   • edit → paste your corrected AFTER text"

   If the user replies "edit", accept their text, show the updated diff, and ask again.

Phase 5 — Apply patch and commit

Only runs after the user replies "yes".

1. Apply the patch: replace the exact BEFORE text with the AFTER text in ~/.claude/skills/<skill>/SKILL.md.
2. Read the file back and confirm the AFTER text is present. If not, stop and report: "Patch verification failed — file may not have been written."

3. Run these git commands from ~/.claude/:

   cd ~/.claude
   git add skills/<skill>/SKILL.md
   git commit -m "skill(<skill>): improve <section> — <one-line reason>"

4. If git commit fails, say: "Git commit failed: <error>. File was updated on disk but not committed. Please commit manually." Do not block on this.

Phase 6 — Resolve feedback entries

In ~/.claude/skill-data/feedback-log.md, for every entry whose (skill, section) pair was addressed by the patch, change status: open to status: resolved. Do not delete entries. Do not modify any other field.

Phase 7 — Final report

Run complete.

Patches applied: <count>
Skills improved: <list>
Feedback entries resolved: <count>
Feedback entries still open: <count> (<reason if any were skipped>)

Next: continue using your skills. Log issues with $skill-feedback.
Run $skill-improver again after accumulating more feedback.

Rules

• Never apply a patch without explicit "yes" from the user in Phase 4.
• Never modify feedback-log.md except to change status in Phase 6.
• Never touch any domain skill that has zero actionable pairs.
• Always commit after a successful patch. Never leave an uncommitted change.
• Process one skill at a time: complete all phases for skill A before starting skill B.
• Never improve skill-feedback or skill-improver through this flow.