agentleFS
mdsmithaustinmdsmithaustin/pstack/skills/spec-probes/SKILL.md

spec-probes

Skill9 starsChanged 37 days agoMITView on GitHub

Apply when writing or reviewing a spec, requirements, acceptance criteria, user stories, or tickets before any code exists, and when deriving acceptance tests from a spec you cannot change (fixed-spec mode). Two passes: an edge probe over a closed taxonomy of data-shape boundaries, and a prohibition probe for what the feature must never silently become. Every raised item is resolved, dismissed with a reason, or carried as an explicit gap.

The file

We split this skill into the rules it gives an agent

1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Write the backstop and judgment tags into the spec, or into the reply in fixed-spec mode.

    line 61

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Spec probes

A verifier only checks assertions that exist, and it is confidently wrong about edges the spec never stated. Run these two passes over each requirement before code, in this order.

Pass 1: edge probe

Classify each requirement's data shape: numeric-range, collection, text, stateful, io. The shape also includes example when the spec gives the requirement a worked example, and variants when the requirement names several variants, such as formats, languages, or strategies. It includes look-alike when the requirement defines a term that looks like another term in the spec. Then raise only the categories whose shapes intersect it.

CategoryShapesAsk
Unit / cardinalitycollection, statefulWhat counts as one item where the spec counts, numbers, groups, or joins? Take the unit the spec's noun names. Test an input where the plausible units differ: two outer items of two parts each, a header line present, two occurrences in one window.
Boundary valuesnumeric-rangeWhat happens exactly at each min, max, or threshold, and one step either side?
AdjacencycollectionWhen two things are exactly equal or just touch, do they merge, collide, or stay separate?
Empty / degeneratecollection, textWhat is the result for empty, single-element, or null input?
EncodingtextWhose definition of length and equality applies: bytes, code points, grapheme clusters, normalized form?
Ordering / stabilitycollectionWhen elements compare equal, is output order specified and stable?
Precision / overflownumeric-rangeWhere can rounding, tie-breaking, precision loss, or overflow occur, and what is the exact contract: half-up or half-even, floor or truncate?
IdempotencystatefulWhat happens if this runs twice on the same input?
Concurrency / effect orderstateful, ioIf interrupted or run in parallel, what is guaranteed?
Example valuesexampleAn example's output is exact: expect it character for character, never through a parser or normalizer, in the format the examples show where prose allows several. Its input values, and whatever else holds in it that no rule states, such as the order its operations finish, restrict nothing. Add one test where such a property does not hold when a requirement or today's behavior gives its result. Otherwise carry the edge as unresolved.
Variant coveragevariantsDoes each raised edge behave the same in every named variant? Probe it in each variant, or record why checking one variant covers the others.
Look-alike termslook-alikeWhere the spec defines two similar terms differently, which input gets different results from them? Expect both results, so a test fails whichever term is implemented like the other.

A requirement with prose but no matching shape gets one soft "unclassified, review by hand" item, not silence.

The defect this catches: "one reminder per overdue loan", tested with a member who has one loan of one book, passes whether reminders go out per member, per loan, or per book.

Pass 2: prohibition probe

Edge shapes cannot surface "the reminder must not shame the user". For each requirement ask: what could this feature silently become that the author would not want, but the spec does not forbid? Cast wide first; ten raw candidates per requirement is expected.

Then one precision pass:

  • Drop routine engineering: must not mutate its input, must not throw on empty, must not leak a handle. Those belong to pass 1 or to code review.
  • Keep values, fairness, privacy, transparency, and safety items: must not use shaming framing, must not proxy on protected attributes, must not store raw PII in plaintext.
  • Refer, do not mint, anything a dedicated tool already owns (injection, path traversal, data-retention law, generic bias canon): one breadcrumb to the security or compliance step, then stop.

Two or three bespoke items per feature is the normal yield. A pure utility yields zero, and that is the right answer, so a non-empty list always carries signal.

Resolve every raised item

Each item ends in exactly one state:

  • resolved: the spec or today's behavior states the answer (explicit, checked by an ordinary test); or a held-out or property-based test will stand in for an edge you know but cannot fully write down (backstop); or, for a prohibition that needs human judgment rather than a test, judgment.
  • dismissed, with a reason. "N/A, input is a bounded enum, no boundary exists" is valid; silence is not. The reason is the audit trail.
  • unresolved: carried forward as an explicit gap the plan must surface, never dropped silently.

A new rejection needs a requirement that states the restriction. A rejection is an expected result that rejects, errors on, or reinterprets an input. Expect one only where a requirement states the restriction, and name that requirement in the criterion. A stated type, range, or format is such a requirement, so expect an input that breaks it to be rejected. The rule holds on a new field and on input the program accepts today, even when a worked example shows only one form of the value or a sibling field has the rejection.

  • Without such a requirement, expect the result the spec text gives the input, or the result the program gives it today. A new field has no result today, so where the spec text gives none, write no criterion for the input and carry the edge as unresolved.
  • Pair each rejection you test with the nearest valid input its definition allows, such as a leading zero or a single column, and expect it accepted. A validator that fails the pair is wider than its definition.

Name the source of each criterion you write from a spec (a requirement, a spec sentence you quote, or the current behavior it keeps), and leave an edge with no source unresolved with no criterion. Text you add to the spec during the task is not a source. A criterion that answers a pass 2 prohibition and has no source gets tagged assumption.

In fixed-spec mode, only what the spec states gets a test. You are in it whenever you derive tests or criteria from a spec you cannot change. explicit then needs a narrower source: a spec sentence or example you quote, or current behavior you name. An answer you supply yourself, to an edge or a prohibition, is an assumption: write no criterion or test for it, keep the item unresolved, and have the code take the most permissive behavior.

Write the backstop and judgment tags into the spec, or into the reply in fixed-spec mode. A reviewer later abstains on those; without the tag it cannot see its own blind spot.

Reply: per requirement, the items raised and their states; then each assumption; then the coverage count: applicable, resolved, unresolved, and how many are backstop or judgment.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.