Skill Creator

A skill for creating new skills and iteratively improving them.

At a high level, the process of creating a skill goes like this:

• Decide what you want the skill to do and roughly how it should do it
• Write a draft of the skill
• Create a few test prompts and run claude-with-access-to-the-skill on them
• Evaluate the results
  • which can be through automated evals, but also it's totally fine and good for them to be evaluated by the human by hand and that's often the only way
• Rewrite the skill based on feedback from the evaluation
• Repeat until you're satisfied
• Expand the test set and try again at larger scale

Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through these stages. So for instance, maybe they're like "I want to make a skill for X". You can help narrow down what they mean, write a draft, write the test cases, figure out how they want to evaluate, run all the prompts, and repeat.

On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop.

Of course, you should always be flexible and if the user is like "I don't need to run a bunch of evaluations, just vibe with me", you can do that instead.

Cool? Cool.

Building Blocks

See references/building-blocks.md for details on Eval Run, Grade Expectations, Blind Compare, and Post-hoc Analysis building blocks.

Environment Capabilities

Check whether you can spawn subagents — independent agents that execute tasks in parallel. If you can, you'll delegate work to executor, grader, comparator, and analyzer agents. If not, you'll do all work inline, sequentially.

This affects which modes are available and how they execute. The core workflows are the same — only the execution strategy changes. Environment-specific mechanics (Claude.ai inline runs, Cowork static viewer/feedback) are in references/environments.md — read it when running outside Claude Code.

Mode Workflows

Building blocks combine into higher-level workflows for each mode:

See references/mode-diagrams.md for detailed visual workflow diagrams.

Task Tracking

Use tasks to track progress when the task tools are available (since v2.1.268 they are on by default only on an allowlist of older models — Claude 3.x, Opus 4–4.7, Sonnet 4–4.6, Haiku 4.5 — and off on everything newer or unrecognized; see the availability section in references/task-tracking.md). See references/task-tracking.md for lifecycle, creation, and stage progression details.

Architecture

The coordinator (this skill):

1. Asks the user what they want to do and which skill to work on
2. Determines workspace location (ask if not obvious)
3. Creates workspace and tasks for tracking progress
4. Delegates work to subagents when available, otherwise executes inline
5. Tracks the best version (not necessarily the latest)
6. Reports results with evidence and metrics

Agent Types

Communicating with the user

The skill creator is liable to be used by people across a wide range of familiarity with coding jargon. If you haven't heard (and how could you, it's only very recently that it started), there's a trend now where the power of Claude is inspiring plumbers to open up their terminals, parents and grandparents to google "how to install npm". On the other hand, the bulk of users are probably fairly computer-literate.

So please pay attention to context cues to understand how to phrase your communication! In the default case, just to give you some idea:

• "evaluation" and "benchmark" are borderline, but OK
• for "JSON" and "assertion" you want to see serious cues from the user that they know what those things are before using them without explaining them

It's OK to briefly explain terms if you're in doubt, and feel free to clarify terms with a short definition if you're unsure if the user will get it.

Creating a skill

Capture Intent

Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill the gaps, and should confirm before proceeding to the next step.

Create a skill when you keep pasting the same instructions, checklist, or multi-step procedure into chat, or when a section of CLAUDE.md has grown into a procedure rather than a fact.

Triage first — is this actually a skill? A skill is on-demand, gated by a decision point Claude can skip, so it fits discrete, invocable workflows ("do X, then Y, then verify" — the "would I write a function for it?" test). It is a poor fit for always-on passive knowledge (framework/API surface, ever-present conventions, code-style rules) — that belongs in CLAUDE.md/AGENTS.md, which loads every turn with no trigger risk. Practitioner evals repeatedly find description-only skills failing to fire on a large fraction of relevant prompts. So: if the user needs something to apply every time, steer it to CLAUDE.md or a hook; reserve the skill format for workflows they (or Claude) explicitly trigger. The two can coexist — a rule can live in CLAUDE.md and a review skill can reference it. Other non-skill answers — an MCP server (reach an external system), an output style (a voice/format for every reply), a mod (UI pane, instant command, rewriting a tool call), /goal or a Stop hook (keep going until a check passes): the "Which form?" table in references/orchestration-guide.md maps each need to its form.

1. What should this skill enable Claude to do?
2. When should this skill trigger? (what user phrases/contexts)
3. What's the expected output format?
4. Should we set up test cases to verify the skill works? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art) often don't need them. Suggest the appropriate default based on the skill type, but let the user decide.

Interview, Research & House Style

The interview is adaptive: a silent triage gate routes simple skills (advise/format/summarize, one pass) to a single AskUserQuestion screen, and complex ones (fan-out, MCP/API, cron, multi-file mutation, mid-run interaction) to a staged flow — intent, research, architecture, guardrails, evals. Follow references/creation-interview.md; it also defines the "just vibe" degradation for users who don't want process.

Research runs on two tiers (references/research-protocol.md): always verify frontmatter fields and orchestration features against the local Claude Code docs mirror before drafting; offer web/library research (library-docs MCP, web search) as an interview option — never launch it unasked.

If references/house-style.md exists, read it in full at the start of every Create run — it carries machine-local conventions (trigger format, allowed-tools form, heavy-skill patterns) that override the generic defaults in this file.

Initialize

Run the initialization script:

scripts/init_skill.py <skill-name> --path <output-directory>

This creates:

• SKILL.md template with frontmatter
• scripts/, references/, assets/ directories
• Example files to customize or delete

Fill SKILL.md Frontmatter

Based on interview, fill in the required fields and relevant optional fields:

• name: Command name (kebab-case, max 64 chars). In a personal or project skill, name sets the command you type and the directory name invokes it too (deploy-staging/ + name: deploy → /deploy and /deploy-staging). In a plugin skill, name replaces the command's last segment and the plugin prefix stays (v2.1.216+); in a plugin-root SKILL.md, name supplies the final segment. Keep name == directory basename when the skill must travel — the portable path requires it (§1.1 of references/frontmatter-reference.md). Every skill is also a slash command (/<name>, or /<plugin>:<name>) and appears in the / menu unless user-invocable: false. Check /skills for collisions and avoid reserved and built-in command names — the canonical list lives in references/description-optimization.md §"Naming the skill" (it also covers names that silently don't load, and why a project/user skill named verify runs before every commit).
• description: What the skill does — this is the primary triggering mechanism. Front-load key use cases: the combined description + when_to_use text is truncated at 1,536 characters in the skill listing. Include both what the skill does AND specific contexts for when to use it. Claude tends to "undertrigger" skills — make descriptions a little "pushy" (e.g., "Build dashboards for internal data. Use this skill whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of company data, even if they don't explicitly ask for a 'dashboard.'")
• when_to_use (optional): Additional trigger context — phrases, example requests. Appended to description in the skill listing and counts toward the 1,536-char cap. Use description for WHAT it does and when_to_use for WHEN to invoke it.

Optional fields: invocation control (when_to_use, arguments, argument-hint, disable-model-invocation, user-invocable), execution (allowed-tools, disallowed-tools, model, effort, context/agent/background, paths, shell), session hooks (hooks — registered on invocation and kept for the rest of the session, not dropped when the skill finishes), portability metadata (license, metadata, compatibility). Full field table, semantics, and version notes: references/frontmatter-reference.md §1; quick schema: references/schemas.md.

Portability check: only six fields — name, description, license, compatibility, metadata, allowed-tools — survive outside Claude Code (claude.ai uploads, the Skills API, .skill packaging, enabling a personal skill for Cowork/cloud sessions). Anything else fails those paths with a hard Unexpected key(s) error. If the skill must travel, restrict the frontmatter to those six; scripts/quick_validate.py warns and scripts/package_skill.py refuses. Details: references/frontmatter-reference.md §1.1.

Invocation control: By default, both user and Claude can invoke a skill. Set disable-model-invocation: true for user-only skills (e.g., dangerous operations; also blocks preload into subagents and scheduled-task runs). Set user-invocable: false for Claude-only background knowledge skills that shouldn't appear in the / menu. Matrix and takeaways: references/frontmatter-reference.md §2.

Skills are discovered from .claude/skills/ in the starting directory and every parent directory up to the repo root. Directories added via --add-dir also have their .claude/skills/ loaded automatically; when an added skill's name clashes with an existing one it runs as /<folder>:<name> (v2.1.309+, references/frontmatter-reference.md §1 name row). Useful for monorepo setups.

skillOverrides (settings-based visibility)

Control skill visibility from settings.json without editing SKILL.md — values "on"/"name-only"/"user-invocable-only"/"off"; the /skills menu writes it for you. Caveats (managed-settings-only propagation, Remote Control/SDK hiding, plugin skills excluded) and permission rules (Skill(name) / Skill(name *)): references/frontmatter-reference.md §8.

Skill Writing Guide

Anatomy of a Skill

skill-name/
├── SKILL.md (required)
│   ├── YAML frontmatter (name, description required)
│   └── Markdown instructions
└── Bundled Resources (optional)
    ├── scripts/    - Executable code for deterministic/repetitive tasks
    ├── references/ - Docs loaded into context as needed
    └── assets/     - Files used in output (templates, icons, fonts)

Writing the body

Read references/skill-writing-craft.md when drafting or rewriting SKILL.md content. It carries: what NOT to include (no README/CHANGELOG — skills are for agents), progressive disclosure (three-level loading, the <500-line rule, domain organization by variant), the Principle of Lack of Surprise (no malware/misleading skills), and body-writing craft — imperative form, output-format and example patterns, affirmative directives, placing rules where they fire, trimming what the model already knows, and reaching for a hook instead of prose for must-happen steps.

Immediate Feedback Loop

Always have something cooking. Every time user adds an example or input:

1. Immediately start running it - don't wait for full specification
2. Show outputs in workspace - tell user: "The output is at X, take a look"
3. First runs in main agent loop - not subagent, so user sees the transcript
4. Seeing what Claude does helps user understand and refine requirements

Claude Code watches skill directories — file edits are detected without restarting. However, already-loaded skill content in the current conversation is NOT updated — re-invoke the skill to pick up changes. The iteration loop is: edit SKILL.md → re-invoke (or run /reload-skills, v2.1.152+, to re-scan all skill directories without restarting) → re-test. Only a brand-new top-level skills directory that didn't exist at session start needs /reload-skills or a restart to be picked up.

Test Cases

After writing the skill draft, come up with 2-3 realistic test prompts — the kind of thing a real user would actually say. Share them with the user: [you don't have to use this exact language] "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then run them.

If the user wants evals, create evals/evals.json with this structure:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User's task prompt",
      "expected_output": "Description of expected result",
      "files": [],
      "expectations": [
        "The output includes X",
        "The skill correctly handles Y"
      ]
    }
  ]
}

You can initialize with scripts/init_json.py evals evals/evals.json and validate with scripts/validate_json.py evals/evals.json. See references/schemas.md for the full schema.

Transition to Automated Iteration

Once gradable criteria are defined (expectations, success metrics), Claude can:

• More aggressively suggest improvements
• Run tests automatically (via subagents in the background if available, otherwise sequentially)
• Present results: "I tried X, it improved pass rate by Y%"

Baseline gate. A skill earns its place only if it beats the no-skill baseline. Many shared skills make output worse or never fire. So when evals exist, run the same prompts with and without the skill and require a demonstrable improvement before calling it done — a skill that matches vanilla is pure context cost. Keep the grader/judge independent from whatever produced the output (don't let the same context grade itself), and treat small score deltas as noise rather than signal.

Choosing an Orchestration Architecture

When the skill being created involves more than single-pass inline work — fan-out over many items, background or scheduled runs, parallel file mutation — pick the execution architecture during the interview, before drafting. Seven options: plain inline, forked subagent (context: fork — runs in the background by default since v2.1.218, with a narrower tool set and only partial /rewind coverage; background: false restores foreground semantics), direct subagents (hub-and-spoke), thin skill + saved workflow, agent teams (experimental, env-gated), hooks as a cross-cutting enforcement layer, and scheduled/cron. The constraint that eliminates options fastest: workflows and cron runs accept no mid-run user input — anything interactive must happen before the fan-out launches or after it returns. A forked skill can't ask either: every subagent loses AskUserQuestion and Workflow.

See references/orchestration-guide.md for frontmatter signatures, the decision matrix, resolution order, and worked examples. Where a stage needs its own model or effort, read references/model-routing.md before setting model:/effort: anywhere.

Package and Present

After creating or improving a skill, package it:

python -m scripts.package_skill <path/to/skill-folder>

Direct the user to the resulting .skill file path so they can install it. This is the quick, single-recipient path. For skills that need to be shared, versioned, or published, wrap them in a plugin + marketplace instead — see references/plugin-packaging.md (layout, plugin.json/marketplace.json, version-bump rules, claude plugin validate --strict).

Advanced Skill Features

These features extend what skills can do. See references/frontmatter-reference.md for full documentation and examples.

String Substitutions

Variables replaced at load time: $ARGUMENTS / $N / $name (arguments), ${CLAUDE_SKILL_DIR}, ${CLAUDE_SESSION_ID}, ${CLAUDE_EFFORT}, ${CLAUDE_PROJECT_DIR}. Full table, examples, and edge cases (auto-append when $ARGUMENTS is absent, unmatched $N preserved verbatim): references/frontmatter-reference.md §3.

Dynamic Context Injection

Skills can embed shell command output directly into their content at load time. Commands execute before content reaches Claude — Claude only sees the substituted output. Useful for injecting git diffs, file listings, API responses, etc. See references/frontmatter-reference.md section 4 for syntax and examples.

Note: If disableSkillShellExecution: true is set in settings, !`command` placeholders in user/project/plugin/additional-directory skills are replaced with [shell command execution disabled by policy]. Bundled and managed skills are exempt. Warn users if their skill relies on dynamic context.

Reasoning depth (effort, not ultrathink)

ultrathink in a user prompt asks for deeper reasoning on that one turn (a Claude Code keyword; "think hard" is plain text): Claude Code always adds an in-context instruction, and since v2.1.311 the word typed at the interactive terminal (VS Code since v2.1.316) can also raise that turn to xhigh — only on Opus 5.5, Sonnet 5.5, Haiku 5.5 or Fable 5.1 through Anthropic's API, not on Bedrock and never above CLAUDE_CODE_EFFORT_LEVEL or maxEffortLevel; the session level stays unchanged. On Opus 5.5 and Fable thinking is always on — for a skill, set depth with effort: in frontmatter, not with prompt lines. See references/frontmatter-reference.md for details, and references/model-routing.md for choosing the level (and why model: on an inline skill costs a cache rebuild while effort: usually doesn't).

Skills as Subagents

To create or manage custom agents, ask Claude or edit .claude/agents/*.md directly (the /agents wizard was removed in v2.1.198).

Set context: fork and agent to run the skill in an independent subagent:

---
name: security-scan
context: fork
agent: general-purpose
---

Preloading Skills into Subagents

Custom agents can preload skills via the skills field in their frontmatter:

---
name: api-developer
skills:
  - api-conventions
  - error-handling
---

Skill Content Lifecycle

When invoked, skill content stays in the conversation for the session. On compaction, each skill retains its first 5,000 tokens (25,000 combined budget, MRU-first). Re-invoke after compaction to restore full content. See references/skill-lifecycle.md for details.

Context Budget

The skill listing is budgeted at a fraction of the model's context window (default 1%); combined description + when_to_use is truncated at 1,536 characters per skill — front-load key use cases. Keep SKILL.md under 500 lines. Budget overrides (skillListingBudgetFraction, skillListingMaxDescChars, SLASH_COMMAND_TOOL_CHAR_BUDGET) and diagnostics (/context, /doctor): references/frontmatter-reference.md §7.

Validation

Run claude plugin validate as the primary validator for frontmatter and plugin structure — it is the canon. python -m scripts.quick_validate is a lightweight authoring guardrail. In its default Claude Code mode it fails only on hard errors (missing description, malformed YAML, over-limit lengths) and warns on everything else, including unknown keys — there a warning is a prompt to double-check the docs, not a blocker. In --portable mode (the mode package_skill.py runs) the rules are stricter: portability violations — any field outside the portable allowlist, name ≠ directory name — become hard errors, because the upload path rejects them.

Improving a skill

Iteratively optimize an existing skill using the building blocks (run, grade, compare, analyze).

IMPORTANT: Before running improvement iterations, read the full documentation:

Read references/improve-mode.md   # Complete Improve workflow
Read references/schemas.md        # JSON output structures

Key questions to ask: Which skill? How much time? What's the goal?

Without subagents, use single runs per iteration and inline grading. See references/improve-mode.md for details.

Eval Mode

Run individual evals to test skill performance and grade expectations.

IMPORTANT: Before running evals, read the full documentation:

Read references/eval-mode.md      # Complete Eval workflow
Read references/schemas.md        # JSON output structures

Which eval system. A skill that ships in a plugin → the native claude plugin eval (isolated runs, built-in no-plugin baseline; skillOverrides can't switch plugin skills off for a baseline). A personal or project skill → this mode's evals/evals.json loop. The two formats don't read each other's files. To keep a built-in tool such as Agent out of native runs, pass --disallow-tools Agent or set the case field disallowed_tools (v2.1.299+). Recipe for both: references/eval-mode.md §"Native plugin eval".

Use Eval mode when:

• Testing a specific eval case
• Comparing with/without skill on a single task
• Quick validation during development

The workflow: Setup → Check Dependencies → Prepare → Execute → Grade → Display Results

Without subagents, execute and grade sequentially in the main loop. Read the agent reference files (agents/executor.md, agents/grader.md) and follow the procedures directly.

Benchmark Mode

Run standardized performance measurement with variance analysis.

Requires subagents. Benchmark mode relies on parallel execution of many runs to produce statistically meaningful results. Without subagents, use Eval mode for individual eval testing instead.

IMPORTANT: Before running benchmarks, read the full documentation:

Read references/benchmark-mode.md # Complete Benchmark workflow
Read references/schemas.md        # JSON output structures

Use Benchmark mode when:

• "How does my skill perform?" - Understanding overall performance
• "Compare two models" - Cross-model comparison (name full model IDs such as claude-sonnet-5-5, never aliases — an alias can be redirected by ANTHROPIC_DEFAULT_*_MODEL and silently compare a model with itself; references/model-routing.md)
• "Has performance regressed?" - Tracking changes over time
• "Does the skill add value?" - Validating skill impact

Key differences from Eval:

• Runs all evals (not just one)
• Runs each 3 times per configuration for variance
• Always includes no-skill baseline
• Analysis runs on the session's model unless an analyzer agent pins one (references/model-routing.md)

Description Optimization

The description field in SKILL.md frontmatter is the primary mechanism that determines whether Claude invokes a skill. After creating or improving a skill, offer to optimize the description for better triggering accuracy.

See references/description-optimization.md for the full 4-step optimization workflow (generate queries, review with user, run optimization loop, apply result).

Workspace Structure

Workspaces are created as sibling directories to the skill being worked on (e.g., skill-name-workspace/).

See references/workspace-structure.md for detailed directory layouts for each mode (Eval, Improve, Benchmark).

Coordinator Responsibilities

See references/coordinator.md for the full coordinator responsibility list.

Delegating Work

See references/delegating-work.md for delegation patterns with and without subagents.

Reference files

The agents/ directory contains instructions for specialized subagents:

• agents/executor.md — How to execute an eval run and produce transcript + outputs
• agents/grader.md — How to evaluate assertions against outputs
• agents/comparator.md — How to do blind A/B comparison between two outputs
• agents/analyzer.md — How to analyze why one version beat another

The eval-viewer/ directory contains the browsable review UI:

• eval-viewer/generate_review.py — generate/serve the review page (Outputs + Benchmark tabs, per-eval feedback); --static for headless environments

The references/ directory has additional documentation:

• references/schemas.md — JSON structures for evals.json, grading.json, benchmark.json, etc. + frontmatter schema
• references/frontmatter-reference.md — Complete frontmatter field documentation, invocation control, string substitutions, hooks
• references/orchestration-guide.md — Which form (skill / hook / mod / MCP / output style / /goal), seven execution architectures, decision matrix, subagents vs agent teams, worked examples
• references/model-routing.md — Model and effort per stage: source priority, cache cost, advisor pairs, full IDs vs aliases, calibrating effort
• references/creation-interview.md — Adaptive Create interview: triage gate, simple/staged paths, orchestration fork, "just vibe" degradation
• references/research-protocol.md — Two-tier research: local docs mirror always, web/library research on request
• references/house-style.md — Machine-local conventions; read in full at the start of every Create run when present
• references/skill-writing-craft.md — What not to include, progressive disclosure, lack-of-surprise principle, body-writing patterns and style craft
• references/sandboxing.md — Sandbox settings that affect skills running bash (network allowlist, credential masking, filesystem isolation)
• references/AUDIT.md — Audit registry: per-file audit dates, SYNC registry of facts duplicated across files, release checklist
• references/integration-testing.md — TESTS.md capability matrix + gate for skills with live external dependencies
• references/plugin-packaging.md — Distributing a skill as a plugin: layout, plugin.json/marketplace.json, version semantics, validation
• references/skill-lifecycle.md — Skill content lifecycle, compaction behavior, re-invocation
• references/agent-authoring.md — When skills create companion agents, skill vs agent frontmatter
• references/building-blocks.md — Eval Run, Grade Expectations, Blind Compare, Post-hoc Analysis
• references/task-tracking.md — Task lifecycle, creation, stage progression
• references/coordinator.md — Coordinator responsibility checklist
• references/delegating-work.md — Delegation patterns with/without subagents
• references/improve-mode.md — Complete Improve workflow (setup, iteration loop, final report)
• references/description-optimization.md — Description optimization 4-step workflow
• references/eval-mode.md — Complete Eval workflow
• references/benchmark-mode.md — Complete Benchmark workflow
• references/environments.md — Claude.ai / Cowork adaptations (inline runs, static viewer, feedback.json)
• references/mode-diagrams.md — Visual workflow diagrams
• references/workspace-structure.md — Directory layouts for each mode

Conclusion

The loop, one more time: scope it (triage → interview → architecture) → draft → run realistic test prompts → evaluate (by hand or with evals) → rewrite → repeat, expanding the test set as the skill stabilizes. Good luck!