agentleFS
craigcossairtcraigcossairt/trellis/.claude/skills/build-verification-skill/SKILL.md

build-verification-skill

Skill5 starsChanged 11 days agoMITView on GitHub

Build this project's verification skill - a small CLI that drives the running app and captures evidence, plus a feature map of what exists and how to reach it - and the CI check that keeps the map current. Use when asked to 'build a verification skill', 'let the agent drive the app', 'make a feature map', 'the agent keeps guessing what a bug report means', or before relying on an agent to reproduce bugs.

  • Deletes or force-pushes

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 6 are distinctive, and 3 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Derive, never hardcode: If jumpable routes are declared in the app's router, read them from there, so the lever and the app cannot drift.

    line 38
  2. Very specific:

    Git Bash (MSYS) rewrites path-like arguments: before a native .exe sees them, so /sdcard/x reaches the tool as C:/Program …

    line 52
  3. Very specific:

    Assert on the argv the stub actually received,: never on a --dry-run display line.

    line 56
  4. Distinctive:

    Doctor: - run before the first drive and after any drive that behaved oddly.

    line 69
  5. Distinctive:

    Start with three to five features.

    line 100
  6. Very specific:

    Every jumpable route is either driven by a feature file (<lever> goto <route>, matched exactly, never as a substring) or listed under …

    line 107
  7. Distinctive:

    Every feature file keeps the four H2s.

    line 111
  8. Distinctive:

    No feature file or journey drives a route that is no longer jumpable.

    line 112
  9. Distinctive:

    When a user-facing feature changes, its feature file changes in the same PR.

    line 127

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Build a verification skill

An agent can only be trusted with a change it can check for itself, and reading the code is not checking it. This skill builds the two pieces that let it check: a lever that drives the real running app and captures proof, and a map that turns a vague report ("the badge is wrong ???" plus a cropped screenshot) into a specific feature, screen and set of commands.

Adapted from create-verification-skill in Lauren Tan's pstack (see CREDITS.md).

The output is a project skill, .claude/skills/verify-<app>/, and one script. Nothing here is done until step 6 has run once against a real build: a verification skill that was never executed is a draft.

1. Build the lever: a CLI, not a document

Prose an agent re-derives every session is not a tool. Write one script (bin/control-app.sh or the project's equivalent) with subcommands:

SubcommandDoes
doctorRead-only: is this instance worth driving? Device or browser present, app installed, which build
goto <route>Jump the running app to a screen, through whatever deep-link or URL mechanism exists
screenshot [path]Capture evidence
dump [path]Capture the UI tree, where the platform exposes one
cleanupRemove this run's scratch; never touch the evidence directory

Design rules:

  • Exit 0 ok, 1 a determined negative, 2 could not tell. 2 never collapses into 0: "I could not check" is not "it is fine".
  • --json on anything an agent parses; --dry-run on anything with a side effect; --help is the canonical surface.
  • Refuse production. The lever drives a non-production build against seeded accounts.
  • Derive, never hardcode. If jumpable routes are declared in the app's router, read them from there, so the lever and the app cannot drift.
  • Errors say what to do instead, not only what failed.

2. Guard every input that crosses a shell boundary

These are the defects a stubbed test suite cannot see, measured on a real device run:

  • A remote shell re-parses its argument list. adb shell <args> (and ssh host <args>) joins argv into ONE string and runs it in the remote shell, so a URL containing & backgrounds the command and runs the rest as a separate one. Quote the value for the remote shell, and make that safe by refusing any input that is not one line of an allowlisted character set before it is used. grep -F treats a pattern containing a newline as several patterns, so an allowlist lookup alone passes "/allowed<newline>';rm -rf /;'".
  • Git Bash (MSYS) rewrites path-like arguments before a native .exe sees them, so /sdcard/x reaches the tool as C:/Program Files/Git/sdcard/x. Export MSYS_NO_PATHCONV=1 for a script whose native-tool arguments are remote paths, then convert LOCAL paths explicitly (cygpath -w, which is absent off Windows).
  • Assert on the argv the stub actually received, never on a --dry-run display line. The display can be quoted correctly while the command that runs is not.
  • Unset inherited environment in the test for any variable the script is supposed to set itself, or the case passes vacuously on a machine that already sets it.

3. Write the skill

.claude/skills/verify-<app>/SKILL.md with these six sections, each naming its subcommand:

  • Launch - the exact commands that build and start a non-production instance, and what "ready" looks like. Prefer a visible marker (an environment ribbon, a banner) over doctor: doctor can confirm an app is installed, not that it is the right build talking to the right backend.
  • Doctor - run before the first drive and after any drive that behaved oddly. A sick instance reports as a broken feature, the most expensive false finding there is.
  • Drive - compute taps from the UI tree, never from a screenshot. When a node is missing but something scrollable is on screen, scroll and re-dump before concluding it is absent. Record platform quirks here (for example: text fields that dump without their label, a keyboard that hides the button you need).
  • Evidence - capture the action AND the resulting state; exercise the real user path, never an internal setter or a seeded shortcut; check side effects (the row written, the message delivered to the other account). Gitignore the evidence directory: screenshots carry personal data.
  • Cleanup - removes scratch only. Afterwards, list the evidence directory and confirm the files are still there; do not assume it.
  • Isolation - can two drivers run side by side? Usually only on different devices AND as different accounts, because state lives on a shared backend. If there is no lock, say so.

4. Write the feature map

.claude/skills/verify-<app>/features/:

  • README.md is the map itself, not an index: baseline preconditions every drive assumes (build, signed-in account, feature gates), a role table of seeded accounts read from the live environment rather than from a doc, the feature list grouped by area, a full-sweep order (least to most state-changing), driving conventions, and an Unmapped routes list where every entry carries a reason.
  • One file per feature, with exactly four H2s in this order: ## Sub-features, ## How to get to it (user POV), ## Driving it with <lever>, ## Gotchas. Fixed headings are what make the map sweepable. Ground every handle in visible text or accessibility labels and cite file:line for every gotcha.
  • multi-surface-journeys.md, read last: flows that need two accounts or two devices. Proof lives on the side that did NOT act.

Start with three to five features. The check in step 5 and the maintenance in step 7 grow it.

5. Keep the map current with a check, not a routine

A map is only useful while it is true, and a doc that claims coverage is not coverage. Write bin/check-feature-map.sh and run it in CI, including when the router file changes:

  1. Every jumpable route is either driven by a feature file (<lever> goto <route>, matched exactly, never as a substring) or listed under Unmapped routes with a non-empty reason.
  2. The unmapped list cannot go stale: a listed route that is also mapped, or no longer jumpable, fails.
  3. Every feature file keeps the four H2s.
  4. No feature file or journey drives a route that is no longer jumpable.

Fail closed: an unreadable router, an empty route set, a missing README, or no feature files is exit 2, never clean. A flag given without its value is refused, not looped on (shift 2 with one argument left shifts nothing in bash). Pin the checker with a mutation-validated suite (/pin-guardrail).

6. Run it once, end to end

Launch, doctor, drive one mapped feature, capture evidence, clean up, and confirm the evidence survived. The first real run is where the defects in step 2 show up; expect to fix the lever. Record each one as a gotcha and pin it with a test that fails when the fix is reverted.

7. Maintain

When a user-facing feature changes, its feature file changes in the same PR. Periodically audit whether each file is still TRUE (strings, taps, accounts) by running its Driving section; the check in step 5 cannot see that.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.