agentleFS
Sign inStart free
SaluteYBSaluteYB/openclaw-skill-eval/skill-eval/SKILL.md

skill-eval

Skill0 starsChanged 8 months agoMITView on GitHub

Evaluate, tune, and regression-check AgentSkills. Use when creating a new skill or improving an existing one and you need structured should-trigger, should-not-trigger, boundary-case, and execution-quality examples. Also use when auditing overlapping skills and sharpening trigger descriptions before release.

We split this skill into the rules it gives an agent

14 rules. 0 are common, 11 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 17 edited versions we read keep these.

  • 11

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill to evaluate another skill before or after changes.

    line 8
  2. Distinctive:

    Use this skill when you need to:

    line 12
  3. Distinctive:

    test boundary cases where two or more skills may conflict

    line 16
  4. Distinctive:

    keep a compact regression log after each improvement round

    line 18
  5. Very specific:

    Create an eval/ folder inside the target skill, or a sibling folder dedicated to that skill.

    line 23
  6. Distinctive:

    Test the skill conceptually or through real use.

    line 27
  7. Very specific:

    If trigger quality is weak, tighten the target skill's description before expanding its body.

    line 29
  8. Distinctive:

    Prefer 8-15 short examples per bucket instead of giant datasets.

    line 34
  9. Distinctive:

    Write examples like real user messages, not synthetic benchmark jargon.

    line 35
  10. Distinctive:

    Use for requests where the target skill is clearly the best match.

    line 44
  11. Distinctive:

    Use for nearby requests that belong to another skill or no skill.

    line 48
  12. Distinctive:

    Use for ambiguous requests.

    line 52
  13. Distinctive:

    Use for post-trigger quality.

    line 56
  14. Very specific:

    Read references/evaluation-patterns.md for compact scoring guidance.

    line 66

“Common” is measured: kept by at least 6 in 10 of the 17 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Skill Eval

Use this skill to evaluate another skill before or after changes. The goal is to reduce black-box guessing by writing down trigger expectations, boundary conflicts, execution checks, and short iteration notes.

Core use cases

Use this skill when you need to:

  • evaluate whether a skill should trigger on realistic requests
  • capture requests that should not trigger the skill
  • test boundary cases where two or more skills may conflict
  • define post-trigger execution quality checks
  • keep a compact regression log after each improvement round

Recommended workflow

  1. Identify the target skill and read its SKILL.md.
  2. Create an eval/ folder inside the target skill, or a sibling folder dedicated to that skill.
  3. Copy the example files from assets/examples/.
  4. Fill in should-trigger.md, should-not-trigger.md, and boundary-cases.md with short, realistic user requests.
  5. Fill in execution-checklist.md with the target skill's quality bar.
  6. Test the skill conceptually or through real use.
  7. Record findings in evaluation-log.md.
  8. If trigger quality is weak, tighten the target skill's description before expanding its body.
  9. Re-test a small representative set after each meaningful change.

Design rules

  • Prefer 8-15 short examples per bucket instead of giant datasets.
  • Write examples like real user messages, not synthetic benchmark jargon.
  • Include at least 3 boundary cases for neighboring skills.
  • Separate trigger quality from execution quality.
  • Record regressions explicitly so old failures do not quietly return.

Evaluation buckets

Should trigger

Use for requests where the target skill is clearly the best match.

Should not trigger

Use for nearby requests that belong to another skill or no skill.

Boundary cases

Use for ambiguous requests. For each case, note which skill should win and why.

Execution checklist

Use for post-trigger quality. Example checks:

  • Did it inspect inputs before acting?
  • Did it verify required tools before claiming success?
  • Did it preserve originals unless overwrite was requested?
  • Did it stay inside the skill boundary?
  • Did it report outputs and tradeoffs clearly?

Use bundled resources

  • Read references/evaluation-patterns.md for compact scoring guidance.
  • Copy files from assets/examples/ to bootstrap a new evaluation pack quickly.

Output expectation

At the end of one evaluation round, produce a short summary containing:

  • target skill
  • main trigger conflicts found
  • description changes made
  • execution issues found
  • current confidence
  • next fixes to try

Download now (ZIP)

  • Other versions19 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 19

    1. boshu2/agentops★ 452same name
    2. IndelibleVivi/skill-field-lab★ 13same name
    3. sunitghub/canon-skills★ 9same name
    4. joris887/exosuit★ 8same name
    5. michalstrnadel/source-to-skill★ 8same name
    6. greglas75/zuvo★ 7same name
    7. aiocean/claude-plugins★ 4same name
    8. ArangoGutierrez/claude-toolkit★ 4same name
    9. sean2077/skills★ 4same name
    10. databricks-industry-solutions/hls-skills★ 3same name

    9 more not listed

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • Used alongsideProjects with it also use postmortem, review and 1 more.

    Other files people use alongside this one.

    Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
    • Hongyu-yuSkill

      postmortem

      Use when analyzing failures, outages, incidents, or negative outcomes, conducting blameless postmortems, documenting root causes with 5 Whys or fishbone diagrams, identifying corrective actions with owners and timelines, learning from near-misses, establishing prevention strategies, or when user mentions postmortem, incident review, failure analysis, RCA, lessons learned, or after-action review.

      Also in 3 projects that use this

    • SentrySkill

      review

      Fixture review skill.

      Also in 4 projects that use this

    • MicrosoftSkill

      handoff

      Compact the current conversation into a handoff document for another agent to pick up.

      Also in 3 projects that use this

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free