agentleFS
web-infra-devweb-infra-dev/rstest/.agents/skills/verify/SKILL.md

verify

Skill514 starsChanged 9 days agoMITView on GitHub

Behavioral verification rules for claiming a change works. Use before reporting any fix/feature as done, when tempted to conclude from typecheck or unit-test results alone, when deciding what evidence a change needs, or when a test result looks suspicious (stale build, flaky pass, snapshot churn).

188 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use instinct-export, instinct-import and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • TheKREATORSSkill

    instinct-export

    Export instincts from project/global scope to a file

    Also in 3 projects that use this

  • TheKREATORSSkill

    instinct-import

    Import instincts from file or URL into project/global scope

    Also in 3 projects that use this

  • AviadPSkill

    risk-assess

    Assess the risk and blast radius of a proposed code change before implementing it. Evaluates CI impact, framework-wide vs localized scope, rollback path, and identifies blindspots. Use when user says "what's the risk", "is this safe", "assess the impact", "risk assess", "blast radius", or before applying framework-wide or CI-affecting changes. Do NOT use for code quality review (use code-quality) or plan validation (use design-validator).

    Also in 3 projects that use this

  • diego-tobalinaSkill

    build-fix

    Fix build and compilation errors systematically. Run when build fails.

    Also in 6 projects that use this

  • AaravChadhaSkill

    secure

    Security review with a confidence gate - a finding exists only with a concrete exploit scenario and a high/medium/low confidence rating; everything else goes to a worth-hardening list, never inflated. Sweeps five surfaces: auth gates, secrets hygiene, injection and unsafe sinks, unsafe deserialization/crypto/transport, and LLM tool-use trust boundaries. Reports only, never fixes. Use when the user asks for a security review or to check vulnerabilities, secrets, or auth.

    Also in 3 projects that use this

  • ArvorCoSkill

    specify

    Create feature specification from natural language description. Use when starting a new feature or creating a spec. Triggers on: create spec, specify feature, new feature spec.

    Also in 10 projects that use this

  • +1 more from TheKREATORS

The file

We split this skill into the rules it gives an agent

3 rules. 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • 0

    Common

    Most of the 167 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    If verification is genuinely impossible (needs real CI, a specific OS, a headed browser you can't run), say so explicitly instead of …

    line 24
  2. Very specific:

    Drive the real binary on the smallest repro: Run the actual rstest CLI the way a user would — an e2e fixture, not an import of internal …

    line 29
  3. Very specific:

    Observe the output, not the summary: Read the reporter output for the specific behavior you changed, and check the exit code (echo $?) when …

    line 30

“Common” is measured: kept by at least 6 in 10 of the 167 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Verify Behavior, not proxies

A change is verified only when you have observed the real behavior change — the actual CLI output, reporter output, or exit code — not when a proxy signal went green. This skill owns exactly one concern: what counts as evidence. How to run things lives in the testing skill; what work a change requires (including the Flip-and-Verify regression protocol) lives in the development skill — don't expect commands or checklists here.

Forbidden proxies

None of these, alone, justify claiming a behavioral change works:

  • Typecheck / lint green. Proves the types compose, not that the behavior changed.
  • Unit tests green. They exercise source via the workspace runner, not the built CLI → runner → reporter pipeline users run.
  • "My new test passes." A test that passes both before and after the fix proves nothing — it must flip (protocol: development → Flip-and-Verify).
  • "The code path is clearly hit." Reading the code and reasoning that it must work is prediction, not observation.
  • Snapshot updated with -u. Regeneration makes tests green by definition; green-after-update is not evidence (update policy: testing → Snapshot policy).
  • Build succeeded. Compiling is not running.
  • A tool accepted your config. Silently-ignored options look identical to working ones — prove the rule/option fires by observing it reject or change something (inject a violation, toggle the option).

If verification is genuinely impossible (needs real CI, a specific OS, a headed browser you can't run), say so explicitly instead of substituting a proxy.

The observation loop

  1. Start from fresh build state. E2E and fixture runs consume built output; if a result contradicts your expectation, suspect a stale or half-finished build before suspecting the code (rebuild procedure: testing → Rebuild before E2E).
  2. Drive the real binary on the smallest repro. Run the actual rstest CLI the way a user would — an e2e fixture, not an import of internal functions (run forms: testing → Running tests).
  3. Observe the output, not the summary. Read the reporter output for the specific behavior you changed, and check the exit code (echo $?) when the change affects pass/fail semantics.
  4. Observe both directions when feasible. See the broken behavior without your change and the fixed behavior with it — a fix you never saw fail is unverified.

What to observe, by change shape

Change touchesMinimum real observation
Core runtime / runner / poolA targeted e2e run plus its exit code, on a fixture that exercises the changed behavior
Reporter / console outputThe actual stdout/stderr the CLI prints, not just assertion results
CLI flags / config optionsTwo runs — with and without the option — confirming the behavior differs
Browser modeA browser e2e run (headless is the default; see testing → Browser E2E)
Adapters (rsbuild / rslib / rspack)A fixture run through the adapter, confirming the transformed config takes effect at runtime
Coverage providersThe emitted report content, not just "the run succeeded"
Watch modeAn actual watch session reacting to a file change, not a one-shot run
Lint rules / hooks / gatesThe gate firing on an injected violation with the intended message, then passing when clean

False-signal gotchas

  • A pass on re-run after a fail may be flakiness, not a fix. Re-run the exact failing command; if results alternate with no code change, report it as flaky rather than fixed.
  • Inexplicable fixture behavior is usually stale state, not logic — stale dist, persistent fixture output, shared cwd (mechanisms and cleanup: testing).
  • Absence of a failure is weak evidence. "It didn't error" only counts if you confirmed the run actually reached the changed code path.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.