agentleFS
sumithrsumithr/sumo-qa/.claude/skills/scaffold-sumo-qa-skill/SKILL.md

scaffold-sumo-qa-skill

Skill7 starsChanged 43 days agoApache-2.0View on GitHub

Scaffolds a new sumo-qa sub-skill end-to-end — SKILL.md skeleton, matching promptfoo eval YAML, and an approaches.md catalogue entry — through a single bundled script. Use this whenever the user wants to add a new sumo-qa-* skill, even if they describe it as "creating a skill", "adding an approach", "starting a new routing branch", or just "writing skills/sumo-qa-X". Enforces the sumo-qa- name prefix and checks for collisions against both the source tree and the installed MCP server, so a clean handoff to the next contributor is always possible.

  • Installs packages

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Add the routing line in skills/sumo-qa-deciding-approach/SKILL.md manually.

    line 56
  2. Very specific:

    Run sumo-qa-validate so the catalogue still parses.

    line 57

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

scaffold-sumo-qa-skill

Drives the four-step recipe for adding a new sumo-qa-<name> sub-skill so all the pieces land together. The deterministic file generation lives in scripts/scaffold.py; this document is the thin guide for gathering inputs and checking the surfaces the script cannot reach (the installed MCP server).

When to use

Trigger this skill whenever the user asks to add a new sumo-qa sub-skill — phrases like "scaffold a new skill", "add a sumo-qa-X skill", "I want a skill that handles Y for QA", or even just "let's start the routing for Z". The user invokes it explicitly with /scaffold-sumo-qa-skill; it doesn't auto-trigger.

Inputs

Gather these from the user up front. If they only give a partial set, ask for the rest in one round of questions — don't drip-feed.

  1. Skill name (without prefix) — kebab-case, descriptive of the user-intent the skill handles. The script prepends sumo-qa- automatically.
  2. One-line description — for the SKILL.md frontmatter description:. Convention across the estate: start with "Use when ..." so the trigger context is obvious.
  3. Approach tag — the kebab-case routing key that lands in knowledge/approaches.md. The user's sumo-qa-deciding-approach skill consults this catalogue when picking which sub-skill to route to.

Verify the surfaces the script can't reach

The script checks for collisions against the source tree (skills/, tests/evals/promptfoo/, knowledge/approaches.md). It can't check the installed MCP server, which serves a frozen-at-install-time view of the catalogues. Before invoking the script, call:

sumo_qa_load_approaches

through the MCP server and confirm the proposed approach tag isn't already shipped. The source tree may have unreleased entries the MCP doesn't yet see, and vice versa — checking both surfaces is what catches both classes of collision.

Run the script

python3 .claude/skills/scaffold-sumo-qa-skill/scripts/scaffold.py \
  --name <kebab-name> \
  --description "Use when <triggering context>." \
  --approach-tag <kebab-tag>

The script exits non-zero with a clear message on any source-tree collision. If that happens, surface the collision to the user verbatim rather than trying to work around it — the namespace policy exists to keep routing unambiguous.

On success, the script prints the three files it created plus the next-step checklist. Pass that summary back to the user so they know what they still need to do by hand.

What the script does and does not do

It writes three files (the SKILL.md skeleton, the eval YAML stub, the approaches.md block) and prints the follow-up checklist. It deliberately does not edit skills/sumo-qa-deciding-approach/SKILL.md — that file has context-sensitive routing tables that resist mechanical patching, and a half-correct edit there breaks routing for every other sub-skill. The contributor adds the routing line by hand using the approach tag they just registered.

After the script runs

Remind the user, in this order:

  1. Flesh out the new SKILL.md via /skill-creator:skill-creator (or /superpowers:writing-skills) — hand it the scaffolded file as the starting point. This is the repo convention per feedback_use_skill_creator_for_skills: skill-creator handles description-trigger optimisation, format consistency with the rest of the estate, and the eval-driven quality loop. Hand-authoring the body by pattern-matching sibling skills drifts on all three and is exactly the failure mode the scaffolder exists to make easier to avoid. Don't skip this step even when the skill feels "obvious" — every skill in the estate ran through skill-creator at least once.
  2. Replace the seed scenario in the new eval YAML with a concrete case before running npm run eval. The placeholder will fail every assertion. The eval YAML stub is intentionally <REPLACE>-marked throughout so this can't be missed.
  3. Add the routing line in skills/sumo-qa-deciding-approach/SKILL.md manually. The script prints the exact markdown row to paste.
  4. Run sumo-qa-validate so the catalogue still parses.
  5. Reinstall the package (pip install -e .) and restart the MCP host before the new approach tag becomes visible to sumo_qa_load_approaches. Until then, source and installed views disagree — that's expected, not a bug.

The scaffold is a starting point, not a finished skill. Don't open a PR with just the scaffold contents; the eval will fail against the placeholder content, and the repo policy (feedback_eval_fixes_target_skill_not_rubric) is to fix the SKILL.md until the eval passes rather than ship with known gaps.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.