Skill Creator

Create and improve skills following the Agent Skills specification. A skill extends agent capabilities with specialized knowledge, workflows, and tools.

When to Use

• User wants to create a skill from scratch, edit, or optimize an existing skill
• Need to run evals to test a skill or benchmark performance
• Even if they just say "create a skill" or "optimize this skill's description"

Core Loop

1. Capture intent - What should the skill do? When should it trigger?
2. Write draft - Create SKILL.md with frontmatter and instructions
3. Create test cases - Realistic prompts users would actually say
4. Run evals via skill-evaluator - Static checks validate definitions; used the opt-in paired runner with an adapter for behavioral evidence. Never describe a static exit 0 as behavioral proof.
5. Review results - Use eval-viewer for human review + benchmarks
6. Iterate - Improve based on feedback until satisfied
7. Optimize description - Fine-tune frontmatter for better triggering
8. Check trigger uniqueness - Grep sibling descriptions for shared trigger phrases; add Not for <sibling> until no collisions remain

Skill Specification

Directory Structure

skill-name/
├── SKILL.md          # Required: metadata + instructions
├── scripts/          # Optional: executable code
├── references/       # Optional: documentation
├── assets/           # Optional: templates, resources
├── agents/           # Optional: subagent instructions
├── eval-viewer/      # Optional: review page generation
└── evals/evals.json  # Required in this template: at least 3 test cases

Frontmatter Fields

SKILL.md Body

• Keep under 250 lines
• Use progressive disclosure: move detailed content to references/
• Include step-by-step instructions, examples, and common edge cases
• Mandatory: Include ## Rationalizations table to counter common agent excuses
• Mandatory: Include ## Red Flags checklist for early warning behaviors

Optimizing Skill Descriptions

Core Writing Principles

1. Use imperative phrasing — "Use this skill when..." rather than "This skill does..."
2. Focus on user intent, not implementation — Describe what the user is trying to achieve
3. Err on the side of being pushy — Explicitly list contexts where the skill applies
4. Keep it concise — A few sentences; max 1024 characters

Testing & Evaluation

5. Design trigger eval queries — Create ~20 realistic prompts (8-10 should-trigger, 8-10 should-not-trigger)
6. Vary should-trigger queries along multiple axes: phrasing, explicitness, detail, complexity
7. Create strong should-not-trigger queries — Use near-misses that share keywords but need something different
8. Run each query multiple times — Model behavior is nondeterministic; run 3 times
9. Use train/validation splits — ~60% train / ~40% validation

The Optimization Loop

10. Evaluate on both sets — Train results guide changes; validation tells if changes generalize
11. Identify failures in train set only — Keep validation results hidden during iteration
12. Revise strategically:
    • Should-trigger failing → broaden scope or add context
    • Should-not-trigger false-triggering → add specificity about what the skill does not do
13. Select best iteration by validation pass rate
14. Check the 1024-character limit

Paired Behavioral Runs (Opt-In)

Static validation (eval-skills.sh, run-evals.py) cannot measure behavior. When behavior matters, use the same evals/evals.json cases with an adapter:

python3 scripts/behavioral-eval.py plan --skill my-skill --cases 1,4
BEHAVIORAL_EVAL_ADAPTER="my-adapter --json" \
  python3 scripts/behavioral-eval.py run --skill my-skill \
  --provider-name <agent> --provider-model <model>

Every case runs with and without the skill, and both sides must grade before a result counts. Negative cases fail on false activation. Statuses are graded/blocked/not_run; exit codes are 0/1/2/3; tokens and cost are unavailable unless an adapter measured them. A baseline is blessed only by an explicit reviewed bless --accept --blessed-by, never by a run. The adapter contract and artifact schema are in skill-evaluator/references/evaluating-skills.md.

Creating Test Cases

Store in evals/evals.json:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User's task prompt",
      "expected_output": "Description of expected result",
      "files": [],
      "assertions": ["The output includes X"]
    }
  ]
}

Test Case Guidelines

• Template-local requirement: Every skill needs evals/evals.json with at least 3 cases. This is repository policy, not an Agent Skills spec requirement.
• Shape: Use a matching skill_name, unique integer IDs, non-empty prompt and expected_output strings, and non-empty arrays of checkable assertion strings. Optional files is an array of paths to existing local input fixtures.
• Realism: Add file paths, personal context, specific details, casual language
• Variety: Mix formal/casual, terse/context-heavy, single-step/multi-step
• Near-misses: Include queries that share keywords but need something different
• Buckets: Optionally tag explicit, implicit, contextual, negative; a tagged set needs a real out-of-scope negative case, not a keyword trap
• Evidence scope: A static audit checks definitions, not model behavior. Use skill-evaluator's paired mode (runner or manual) for behavioral claims; record unavailable measurements as unavailable, never estimated values.

Domain-Specific Verification Skills

When creating a new domain-specific skill, always include a verification checklist. Use skill-evaluator/references/verification-checklist.md as a starting point.

Benefits

• Deep domain knowledge encoded in skills.
• Reduces onboarding time.
• Ensures consistent debugging and verification approach.

See Also

• skill-evaluator — Evaluate and score skills
• intent-classifier — Route requests to appropriate skills

Rationalizations

Red Flags

• Creating a skill without running the eval loop at least once.
• Skipping the description optimization step and shipping a vague or overly broad trigger.
• Hardcoding project-specific paths or values instead of using relative references.
• Omitting the ## Rationalizations or ## Red Flags sections from SKILL.md.
• Reporting a paired improvement from a static exit code or a single unpaired run.

References

• references/guide.md - Templates, examples, and best practices
• references/scripts.md - CLI reference for the bundled scripts
• references/evaluating-skills.md - Evaluating skill output quality
• references/schemas.md - JSON schemas for evals, grading, timing, benchmark
• references/best-practices.md - Best practices for skill creators
• references/output-patterns.md - Common output patterns
• references/workflows.md - Common workflow patterns
• agents/grader.md - Subagent for grading eval assertions
• agents/analyzer.md - Subagent for analyzing benchmark results
• agents/comparator.md - Subagent for blind A/B comparison
• eval-viewer/generate_review.py - Generate HTML review page from results
• assets/eval_review.html - Self-contained eval set query review tool

Registration and Standards

How to register a skill in AGENTS.md

1. Catalog Update: Add the skill to the "Skills" section in AGENTS.md following the alphabetical order within its category.
2. Docs Sync: Add the skill to agents-docs/skills-reference.md.
3. Registry Update: Run ./scripts/update-agents-registry.sh if applicable.
4. Maintenance: Run python3 scripts/generate-skills-readme.py and ./scripts/generate-available-skills.sh to update auto-generated documentation.

Acceptance Criteria Format

Every new skill must meet these criteria before being merged:

• SKILL.md is under 250 lines.
• Frontmatter contains name, description, and template-local category.
• Optional version follows the recommended release-stamp policy.
• Includes ## Rationalizations and ## Red Flags sections.
• Contains at least 3 realistic eval cases in evals/evals.json.
• Successfully passes ./scripts/validate-skills.sh.

Versioning Conventions

• agents-docs/SKILLS.md is authoritative for frontmatter policy.
• version is recommended but optional: it stamps a skill-set release, not an independent per-skill SemVer line. Bump on a skill-set release, not each edit.
• A lagging stamp is legal; do not force-bump it to quiet a diff.
• template_version is optional too; derive it from root VERSION if used.

Scripts

Bundled helpers for scaffolding, packaging, benchmarking, description optimization, and review-page generation.

• references/scripts.md - CLI reference: init_skill, package_skill, aggregate_benchmark, run_loop, eval-viewer/generate_review.py

Voice & Context

• Default: professional + blog
• Reference: voice-profiles skill for definitions and auto-detection.