agentleFS
14ag14ag/claude-skills/skill-creator2/SKILL.md

skill-creator2

Skill0 starsChanged 3 months agoMITView on GitHub

Create, improve, and evaluate Claude skills with a QA-first workflow. Use when users ask to build a skill, improve this skill, refine this one, make this better, run benchmark-only validation, or optimize a skill description for reliable triggering.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

41 rules. About 0 are common, 22 are distinctive, and 19 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈22

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 19

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When the user gives a vague refinement request (for example: "use skill-creator to refine this skill", "make this better"), do not start …

    line 12
  2. Very specific:

    Ensure QA.txt exists in the target skill root.

    line 16
  3. Very specific:

    Run a comprehensive QA pass against that QA.txt.

    line 17
  4. Distinctive:

    If the user gives minimal direction, proceed with the recommended option.

    line 20
  5. Distinctive:

    Use this fast path as the default for underspecified improvement requests.

    line 22
  6. Distinctive:

    Use triage first, then follow the matching workflow.

    line 26
  7. Distinctive:

    If the conversation already includes requirements, extract them before asking new questions.

    line 39
  8. Distinctive:

    Every major feature must be reachable from a concrete user cue.

    line 43
  9. Very specific:

    Before writing or revising prompt text for any skill, read references/prompt-engineering-best-practices.md.

    line 59
  10. Distinctive:

    Use reasoning guidance only when the task benefits from multi-step thinking.

    line 68
  11. Distinctive:

    Add a self-check pattern for evaluative outputs.

    line 69
  12. Distinctive:

    Use XML-style tags or clear delimiters when mixing instructions, examples, context, and dynamic input.

    line 70
  13. Very specific:

    Read references/intake-and-triage.md before drafting or editing.

    line 78
  14. Very specific:

    When writing SKILL.md for the target skill:

    line 89
  15. Distinctive:

    Keep frontmatter valid and description specific enough to trigger reliably.

    line 92
  16. Distinctive:

    Keep instructions imperative and deterministic.

    line 93
  17. Very specific:

    Keep SKILL.md lean and router-like.

    line 94
  18. Very specific:

    Read references/eval-loop.md and execute it as one continuous sequence.

    line 105
  19. Very specific:

    Run python -m scripts.quick_validate .

    line 109
  20. Very specific:

    Treat QA as passing only when every question in the created or edited skill root QA.txt is answered yes.

    line 111
  21. Distinctive:

    Run a negative-trigger smoke test with an obviously unrelated prompt to confirm the skill does not load.

    line 113
  22. Very specific:

    Create realistic eval prompts and save evals/evals.json.

    line 114
  23. Distinctive:

    Run with-skill and baseline runs in parallel where supported.

    line 115
  24. Very specific:

    Use all agents/*.md workflows as part of testing: agents/grader.md for grading, agents/comparator.md for paired blind comparisons, and …

    line 117
  25. Very specific:

    After each edit cycle, run the loop in references/eval-loop.md.

    line 126
  26. Very specific:

    Ensure QA bootstrap was completed before any create/edit/evaluate action: QA.txt exists and missing assets/QA.txt questions were appended …

    line 130
  27. Very specific:

    After each edit, run Comprehensive QA pass #1 against current QA.txt.

    line 132
  28. Very specific:

    Run Comprehensive QA pass #2 against the expanded QA.txt to confirm improvement, no regressions, and preserved objective fit.

    line 134
  29. Very specific:

    Do not treat the QA stage as complete unless every question in the target skill root QA.txt is answered yes.

    line 136
  30. Very specific:

    Keep the target skill's SKILL.md within practical router size limits.

    line 139
  31. Distinctive:

    When the user confirms satisfaction, explicitly ask them for recommendations to improve the skill-creator itself.

    line 146
  32. Distinctive:

    Use this when the user asks whether the new version is actually better.

    line 155
  33. Distinctive:

    After creation or improvement, offer to optimize skill triggering.

    line 159
  34. Distinctive:

    Prefer test-score-selected descriptions over train-only wins to reduce overfitting.

    line 168
  35. Distinctive:

    After every skill creation or edit cycle that the user confirms as complete, packaging is mandatory.

    line 172
  36. Very specific:

    Always call present_files with that path so the user receives a downloadable zip regardless of environment.

    line 181
  37. Very specific:

    Read references/environment-adapters.md for environment-specific behavior:

    line 188
  38. Distinctive:

    Avoid AI fluff and ambiguous language.

    line 223
  39. Distinctive:

    Prefer deterministic steps over vague guidance.

    line 224
  40. Distinctive:

    Use the highest available reasoning effort by default for design-critical work that reduces future user effort.

    line 225
  41. Distinctive:

    Never skip QA just because a request is underspecified.

    line 227

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Skill Creator

A skill for creating new skills and iteratively improving existing skills.

Fast Path: "Improve this skill" with no specs

When the user gives a vague refinement request (for example: "use skill-creator to refine this skill", "make this better"), do not start with broad spec questions. Run this QA-first intake path first.

  1. Identify the target skill path and current name.
  2. Snapshot the current skill before editing.
  3. Ensure QA.txt exists in the target skill root. Append missing baseline checks from assets/QA.txt.
  4. Run a comprehensive QA pass against that QA.txt.
  5. Produce an improvement brief with prioritized issues, expected impact, and risk.
  6. Present 2-3 concrete improvement options and recommend one.
  7. If the user gives minimal direction, proceed with the recommended option.

Use this fast path as the default for underspecified improvement requests.

Workflow Map

Use triage first, then follow the matching workflow.

SituationPath
Brand new skillCapture Intent -> Write -> Test -> Iterate
Improving an existing skill with clear specsCapture Intent -> Edit -> Test -> Iterate
Improving an existing skill, specs not providedRun QA-first fast path, then continue with scoped improvements
QA or benchmarking only (mature skill)Skip writing. Evaluate as-is first, then propose improvements
User wants to review outputs/results before more editsJump to eval loop review step and launch user review view first
User asks "is the new version actually better?"Run blind comparison workflow
User asks to optimize triggeringRun description optimization workflow
User asks to package/install the skillRun packaging workflow

If the conversation already includes requirements, extract them before asking new questions.

Feature Entry Points

Every major feature must be reachable from a concrete user cue.

User cueFeatureEntry action
"create a skill for X"Skill creationStart capture-intent flow
"improve this skill" (vague)QA-first improvement intakeRun fast path before broad interviews
"edit this part of the skill"Scoped refinementStart existing-skill edit flow
"test/benchmark/QA this skill"QA/benchmark-only modeEvaluate as-is first
"show me outputs to review" or iteration eval in progressUser review workflowGenerate review viewer before more edits unless user opts out
"is new version better?"Blind comparisonRun comparator + analyzer
"improve triggering"Description optimizationRun trigger-eval optimization loop
"package it" / "I want the .skill file"PackagingRun package script and provide artifact path
Environment constraints (no subagents/headless)Environment adapterSwitch to environment-specific reference workflow

Prompt Engineering Reference

Before writing or revising prompt text for any skill, read references/prompt-engineering-best-practices.md.

This is required guidance. All prompt blocks should follow this default shape:

role -> context -> task -> constraints -> output format

Apply with judgment:

  • Use reasoning guidance only when the task benefits from multi-step thinking.
  • Add a self-check pattern for evaluative outputs.
  • Use XML-style tags or clear delimiters when mixing instructions, examples, context, and dynamic input.
  • For long-context tasks, ground outputs in quoted or clearly identified evidence.
  • Surface hidden assumptions and ask for clarification when requirements are underspecified.

Core Workflow

1) Capture Intent and Research

Read references/intake-and-triage.md before drafting or editing.

You must:

  • Clarify the user goal, trigger conditions, output format, constraints, and success criteria.
  • Research professional standards and official documentation relevant to the target domain before writing logic.
  • For skills targeting hobbyist or practitioner communities, research the social platforms where that community is active (forums, subreddits, GitHub topic searches) to surface real pain points and ecosystem gaps that official docs omit. See references/intake-and-triage.md §5 for the platform lookup table and recording requirements.
  • For each new or refined skill, draft 3 candidate core prompt formulations, pick the best, and record rationale in references/design-log.md.

2) Write or Edit the Skill

When writing SKILL.md for the target skill:

  • Preserve skill name and directory when refining an existing skill.
  • Keep frontmatter valid and description specific enough to trigger reliably.
  • Keep instructions imperative and deterministic.
  • Keep SKILL.md lean and router-like. Move heavy implementation details to references/.
  • For every file placed in scripts/, add an entry in SKILL.md that states when to run it, the exact command, how to interpret its output, and what to do if it fails.

Progressive disclosure standard:

  1. Metadata (name, description) should define what and when.
  2. SKILL.md should route to the right reference files.
  3. Scripts and references should hold heavy details and repeatable procedures.

3) Test and Evaluate

Read references/eval-loop.md and execute it as one continuous sequence.

The test/eval workflow includes:

  • Run python -m scripts.quick_validate . from the target skill root.
  • Immediately run mandatory QA against that skill's QA.txt.
  • Treat QA as passing only when every question in the created or edited skill root QA.txt is answered yes.
  • Run a negative-trigger smoke test with an obviously unrelated prompt to confirm the skill does not load; record the prompt and outcome in the workspace.
  • Create realistic eval prompts and save evals/evals.json.
  • Run with-skill and baseline runs in parallel where supported.
  • Draft and refine assertions while runs execute.
  • Use all agents/*.md workflows as part of testing: agents/grader.md for grading, agents/comparator.md for paired blind comparisons, and agents/analyzer.md for benchmark and post-comparison analysis.
  • Grade outputs, aggregate benchmark data, and generate the review viewer.
  • Trigger user review with eval-viewer auto-submit mode and wait for completion before additional edits.
  • Collect user feedback and continue iteration.

4) Improve and Iterate

After each edit cycle, run the loop in references/eval-loop.md.

Required quality gates before presenting changes:

  • Ensure QA bootstrap was completed before any create/edit/evaluate action: QA.txt exists and missing assets/QA.txt questions were appended as-is.
  • After each edit, run Comprehensive QA pass #1 against current QA.txt.
  • Then review description quality and append missing skill-specific QA checks.
  • Run Comprehensive QA pass #2 against the expanded QA.txt to confirm improvement, no regressions, and preserved objective fit.
  • Do not treat the QA stage as complete unless every question in the target skill root QA.txt is answered yes.
  • Feature discoverability and integration checks for new phases.
  • Keep the target skill's SKILL.md within practical router size limits.

Stop when:

  • The user confirms satisfaction, or
  • Feedback is consistently empty, or
  • Additional iterations stop producing meaningful gains.
  • When the user confirms satisfaction, explicitly ask them for recommendations to improve the skill-creator itself.

Advanced: Blind Comparison

For rigorous A/B evaluation between two skill versions, use:

  • agents/comparator.md for blind output comparison
  • agents/analyzer.md for post-hoc winner/loser analysis

Use this when the user asks whether the new version is actually better.

Description Optimization

After creation or improvement, offer to optimize skill triggering.

Read references/description-optimization.md.

Important rules:

  • Description must describe concrete situations, not just capabilities.
  • Include at least one realistic example trigger phrase.
  • Evaluate both should-trigger and should-not-trigger cases.
  • Prefer test-score-selected descriptions over train-only wins to reduce overfitting.

Packaging and Delivery

After every skill creation or edit cycle that the user confirms as complete, packaging is mandatory. Run:

python -m scripts.package_skill <path/to/skill-folder> /mnt/user-data/outputs

This produces a .zip file at /mnt/user-data/outputs/<skill-name>.zip.

Always call present_files with that path so the user receives a downloadable zip regardless of environment. Do not skip this step or leave it as an offer the user must accept. If the packager returns an error, report it and fix the underlying validation failure before re-running.

Environment Adapters

Read references/environment-adapters.md for environment-specific behavior:

  • Claude Code (subagents and full loop)
  • Claude.ai (no subagents)
  • Cowork/headless environments

The high-level workflow remains the same; only execution mechanics differ.

Reference Index

Core references

  • references/intake-and-triage.md - triage, QA-first vague-improvement intake, interview flow, design-log requirements
  • references/eval-loop.md - test execution, grading, benchmark aggregation, review viewer, feedback loop, iteration gates
  • references/description-optimization.md - trigger eval set design and optimization loop
  • references/environment-adapters.md - Claude.ai and Cowork execution differences
  • references/file-routing-index.md - complete non-root file routing map for discoverability/QA
  • references/schemas.md - JSON schemas for eval and benchmark artifacts
  • references/prompt-engineering-best-practices.md - required prompt construction rules
  • references/prompt-research-notes.md - supporting research notes

Subagent references

  • agents/grader.md - grading assertions against outputs
  • agents/comparator.md - blind A/B comparison
  • agents/analyzer.md - benchmark and post-hoc analysis

QA baseline

  • assets/QA.txt - baseline QA checklist to merge into target skill QA.txt

Operating Principles

  • Be flexible with user preference while preserving evaluation rigor.
  • Never use emojis.
  • Avoid AI fluff and ambiguous language.
  • Prefer deterministic steps over vague guidance.
  • Use the highest available reasoning effort by default for design-critical work that reduces future user effort. Only trade down when the user explicitly prefers speed.
  • Preserve existing features unless the user requests removal.
  • Never skip QA just because a request is underspecified.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.