agentleFS
Codagent-AICodagent-AI/agent-skills/skills/prepare-acceptance/SKILL.md

prepare-acceptance

Skill31 starsChanged 6 months agoMITView on GitHub

Explores an implemented change to find defects its green test suite missed, exercises every user-visible requirement the change adds or modifies at least once, sizes further exploration to what actually changed, reports findings without fixing them, waits for current-head CI, and prepares evidence for human review when preparing a change for acceptance, gathering review evidence, performing exploratory or human-style testing, or invoking codagent:prepare-acceptance.

We split this skill into the rules it gives an agent

24 rules. About 0 are common, 19 are distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈19

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the implementation, the diff, and the tests to decide where to look.

    line 23
  2. Very specific:

    Before creating files, check the evidence directory and any caller-provided scratch_dir.

    line 40
  3. Very specific:

    When the caller provides an allowed scratch_dir, put every temporary file, clone, fixture project, seeded data, and server working folder …

    line 48
  4. Distinctive:

    Read the tests covering the changed surface, not the whole suite.

    line 102
  5. Distinctive:

    What is structurally unreachable under the fixture strategy?: Look for a mismatch between what the code reads and what the fixture is.

    line 107
  6. Distinctive:

    Interaction of two independently-correct changes: Ask which two accepted behaviors have never been exercised in the same run.

    line 120
  7. Distinctive:

    Keep a candidate only if it clears all three:

    line 131
  8. Distinctive:

    A mock could not settle it: If reading the code answers the question, read it and report a code-reading finding.

    line 134
  9. Distinctive:

    Predict before running: Write the expected result first, in specifics: how many records, which values, what exit status.

    line 158
  10. Distinctive:

    When any run you make or observe during this pass fails, whether a coverage-floor item, an exploration step, or an acceptance case an older …

    line 170
  11. Very specific:

    If the evidence points to anything the change owns, including its fixtures, smoke configs, pins, or defaults, record a defect in …

    line 179
  12. Distinctive:

    Do not stop until every floor item is exercised or named as a limitation with its reason.

    line 195
  13. Distinctive:

    If every step passed and nothing surprised, re-read Step 0.

    line 208
  14. Very specific:

    Write <evidence-directory>/exploration-log.md: each floor item with where it was exercised or why it was not, the exploration budget and …

    line 224
  15. Very specific:

    After every pass, overwrite <evidence-directory>/acceptance-tested-revision.txt with the exact SHA just tested.

    line 230
  16. Distinctive:

    If that file is missing, or names a SHA that does not resolve in this repository, report it as an impediment and explore the change as a …

    line 244
  17. Distinctive:

    Read the prior exploration log as history, meaning "this area already behaves this way," never as coverage, meaning "this area is signed …

    line 247
  18. Distinctive:

    Before reporting ready, confirm every referenced path exists, that any scratch folder created by this skill was removed, and that no …

    line 281
  19. Distinctive:

    Do not fix defects, modify tracked source or approved artifacts, commit, push, or alter PR metadata.

    line 287
  20. Distinctive:

    Do not run automated suites, linters, or the Validator.

    line 289
  21. Distinctive:

    Do not change approved requirements, scope, or design to make something pass.

    line 290
  22. Distinctive:

    Do not obtain human acceptance, mark the PR ready, merge, archive, or release.

    line 291
  23. Distinctive:

    Do not fuzz or build an exhaustive edge-case matrix.

    line 292
  24. Distinctive:

    Do not treat a successful command exit, an agent summary, or a screenshot alone as proof of behavior.

    line 293

≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Prepare Acceptance

Explore an implemented change for defects its green test suite missed. Produce concise evidence, the findings, and an honest list of what was not exercised, then hand all of it to a human acceptance session.

Target what is true of the running system but asserted nowhere. The suite already covers what someone thought to assert, so re-running its scenarios by hand proves nothing. The failure mode to avoid is not missing a bug; it is producing a long plan that re-walks the happy path and finds nothing, because thoroughness is the easy thing to simulate.

A first pass has two parts. The coverage floor makes sure every user-visible behavior the change promises was seen working in the running product at least once; it is the one place walking specified behavior is required, and it is kept brief. Exploration beyond the floor hunts for what nobody asserted, and most of this skill is about that part.

Targeting versus oracle

Read the implementation, the diff, and the tests to decide where to look. Never use them to decide what correct means. Code is self-consistent, so an oracle drawn from it declares every defect correct. Draw expected behavior from the approved artifacts and from what a user would reasonably expect.

Inputs

Resolve from the caller: approved artifacts, the testing envelope naming authorized effects, credentials, sandboxes, cost, cleanup, and any HT-* obligations reserved for humans; an implementation summary and any implementation session report; an evidence directory; and optional scratch_dir, the caller-provided scratch folder. If approved artifacts or the evidence directory are missing, report that and stop.

Carry HT-* obligations forward as human-review instructions. Never execute them or claim evidence for them.

Scratch space

Before creating files, check the evidence directory and any caller-provided scratch_dir. Resolve relative paths and symlinks for both locations and both comparison roots. Get the worktree root with git rev-parse --show-toplevel. Reject any candidate inside the git worktree or the system temp folder, including a path that resolves there through a symlink. If the evidence directory is rejected, report the unsafe input and stop without writing evidence or scratch files. If scratch_dir is rejected, use the allowed evidence directory as the scratch fallback.

When the caller provides an allowed scratch_dir, put every temporary file, clone, fixture project, seeded data, and server working folder inside it. Never use the system temp folder or a hard-coded temp path for these. Leave cleanup of a caller-provided scratch folder to the caller.

Otherwise, create a new, unique scratch subfolder under the allowed evidence directory for those disposable files and folders. Do not reuse an existing folder or create one under the system temp folder. Keep durable evidence separate from scratch space: exploration-log.md, acceptance-findings.md, screenshots, and acceptance-tested-revision.txt live under the evidence directory, outside the scratch subfolder. Copy any retained evidence out of scratch space, then remove only the folder this skill created before the skill ends on any path, including early stops for defects, revision misalignment, or pending CI. On successful passes, do this before writing the handoff.

Coverage floor

On the first acceptance pass for a change, exercise every user-visible requirement or scenario that the approved specs add or modify at least once, through the product as a user or client would reach it. Derive the floor from the approved specs when the pass starts; do not expect or accept a pre-written case list. The floor is not subject to the budget, the ceiling, or "What earns a run" below: exercise a floor item even when an automated test appears to own it, because the floor checks the running product against the spec, not the suite.

A scenario that describes a failure or error is a floor item too: trigger it, with an injected fault or a scratch fixture if needed, instead of leaving it to the suite.

Group floor items that one realistic journey exercises together, and keep each one brief: predict, act, record the observed values. Anything in the floor you could not exercise is a limitation; name it with its reason. If a floor run failed, back an external limitation reason with the failure evidence you read. If no run was permitted by the approved envelope, such as missing authorized credentials, name that constraint; there is no failure evidence to cite. "Out of budget" is not a reason.

Passes after a fix do not repeat the floor. They are sized from the diff; see "After a fix."

Size exploration beyond the floor

Size from seams moved, not lines changed. A change that only rearranges code inside one function rarely earns a hand-run. A change that moves a seam almost always does, because seams are where automated tests stop being able to see.

Kind of moveWhat it puts at risk
Process boundaryLoad paths, environment inheritance, working directory, how failures surface
Error surfaceWhich failures are now silent, and which rescue is too narrow
Control-flow sourceWhether the new source can express every state the old one could
Data sourceShapes the fixture never contained
Identity or ownershipTwo writers, or none

Budget roughly one exploration step per distinct seam, with a minimum of one and a maximum of eight. The budget covers exploration beyond the coverage floor, never the floor itself. Write the floor items and the exploration budget with its rationale to <evidence-directory>/exploration-plan.md before testing. Do not expand the budget silently later.

Step 0: establish what the suite already owns

Read the tests covering the changed surface, not the whole suite. Answer three questions:

  1. What does each assertion prove, as opposed to appear to prove? When one input feeds two outputs and the test asserts the cheap one, the expensive one is uncovered. Names, ids, counts, and "returned without raising" are the usual cheap ones.
  2. What is structurally unreachable under the fixture strategy? Look for a mismatch between what the code reads and what the fixture is. Code that reads from git needs fixtures that exist in git. Unreachable is fine; unreachable and undocumented means someone assumes it is covered.
  3. Which behavior is newest, and does anything cover it? A fix landing late in review is the likeliest thing in the change to have no test at all.

Where candidates come from

In rough yield order:

  • Seams the change moved, per the table above.
  • Accepted gaps and known limitations. Watching a specified-but-odd behavior actually happen is often the highest-value step in a pass, and it is where a limitation turns out worse than documented.
  • Interaction of two independently-correct changes. Ask which two accepted behaviors have never been exercised in the same run. Each was reviewed alone, so no test owns the pair.
  • Negative space. For each guard added, test the configuration where it cannot fire at all, not only the value it rejects.
  • Representation. Live systems echo data in shapes fixtures never contained: floats for integers, a bare value for a one-element collection, an explicit null for an omitted key.
  • The environment. Which secrets, refs, and versions actually exist where this runs, as opposed to which ones the config names. These findings need no test run at all.

What earns a run

Keep a candidate only if it clears all three:

  1. No automated test owns it, established in Step 0 rather than assumed.
  2. A mock could not settle it. If reading the code answers the question, read it and report a code-reading finding. That is cheaper and more certain than a run.
  3. Being wrong has a consequence someone would care about. "Writes to the wrong project" qualifies. "Logs an odd string" does not.

Drop everything else.

Run order

Order steps so each introduces exactly one new source of reality. When a step fails, the newest dependency is the suspect.

  1. Pure logic against saved fixtures, no network or credentials.
  2. The failure path, with the dependency deliberately absent.
  3. A real read against a sandbox.
  4. A real write against a sandbox.
  5. The same operation twice.
  6. The real runner or CI, which adds checkout, credentials, and environment.

Step 5 is disproportionately productive. A large share of real defects are second-run defects: duplicated records, moved timestamps, replaced identity.

Two habits that find things

Predict before running. Write the expected result first, in specifics: how many records, which values, what exit status. Predict from the approved behavior and from the mechanism separately. Where those two predictions differ, that difference is itself a candidate. A wrong prediction is the most valuable event in a pass, because the model and the system disagree and one of them is a defect.

Print values, not verdicts. Emit the actual identifiers, timestamps, and objects rather than pass or fail. A boolean can only answer the question already thought of. For a UI, read the API for exact values and look at the interface for the mental model; some views are the only proof that a run wrote nothing.

Before blaming the environment

When any run you make or observe during this pass fails, whether a coverage-floor item, an exploration step, or an acceptance case an older plan still lists, read that run's retained failure evidence before ruling it out of scope or recording a host, platform, engine, or external limitation or assumption. Limit this reading to the failed run's retained artifacts: the failing message or warning, its request, response, config, and provenance files, and its log. Check the run's output directory and the evidence directory before concluding that no artifacts were retained. Inspect which CLI, agent, credentials, or endpoint the run actually used. This reading does not count against the exploration budget. Do not debug engines or code outside the change, or re-run with a changed config outside the envelope, to prove a cause.

If the evidence points to anything the change owns, including its fixtures, smoke configs, pins, or defaults, record a defect in <evidence-directory>/acceptance-findings.md instead of a limitation or assumption. For every failed run ruled external, out of scope, pre-existing, or unconfirmed, record in exploration-log.md and any assumptions-ledger entry the evidence path, a short verbatim excerpt of the failing message or warning (with secrets redacted), and the provenance line supporting the attribution: the CLI, agent, credentials, or endpoint the run actually used. If reading the retained artifacts is inconclusive, or no evidence was retained, record the failure as an unresolved assumption with the excerpt (if any) and the available paths. State that the cause is unconfirmed; do not assert an external cause.

For example, a smoke run fails with Not logged in; the run's recorded request shows it used the real CLI rather than the fixture's stub, because the fixture never pinned the agent. That missing pin is a defect in the change.

When to stop

Do not stop until every floor item is exercised or named as a limitation with its reason. For a failed run ruled external, out of scope, or pre-existing, include the evidence path, a short verbatim excerpt of the failing message or warning, and the provenance line supporting that ruling. Beyond the floor, stop when the remaining untested paths are enumerable and named, not when they are empty. "Not exercised" paths deliberately not run have no failure evidence; name why they were excluded without assigning an external cause. End with a sentence like: "Not exercised: the conflict path against live data, and behavior above the deletion cap." That is a complete result. "Everything tested" is never true.

Signs the exploration beyond the floor has drifted into theatre: steps that re-walk floor journeys, re-running a journey the end-to-end suite already walks, hand-testing pure logic with no I/O, more than one step per seam, or a step with no recorded prediction.

If every step passed and nothing surprised, re-read Step 0; the pass probably covered ground the suite already owned. Never manufacture or escalate a finding to compensate. An honest empty pass is a better result than an inflated one.

Report

Record defects in <evidence-directory>/acceptance-findings.md with the tested SHA, affected behavior, expected versus observed, concise reproduction, and evidence paths. Do not fix them, and continue through the remaining independent steps after finding one. Stop early only when a defect makes the rest unreachable or untrustworthy.

Append product, scope, or design ambiguity to the assumptions ledger, defaulting to <evidence-directory>/acceptance-assumptions.md. Never report ambiguity as a defect or silently choose an interpretation. Name the evidence file for each reproduction an entry reports. Keep this pass autonomous and leave those decisions to the human acceptance session.

Write <evidence-directory>/exploration-log.md: each floor item with where it was exercised or why it was not, the exploration budget and rationale, each step with its prediction, its observation, and the resulting finding or nothing, plus the named list of what was deliberately not exercised and why. For failed-run rulings, include the evidence path, short verbatim excerpt, and provenance line as described in "Before blaming the environment."

After every pass, overwrite <evidence-directory>/acceptance-tested-revision.txt with the exact SHA just tested. That file is the diff base for the next pass.

For any UI, store screenshots under <evidence-directory>/acceptance-screenshots/ with the behavior demonstrated, what the reviewer should notice, the tested SHA, and a concise text equivalent. Screenshots support an observation; they are not proof on their own.

After a fix

A fix is a change. Apply this same method to it: read <evidence-directory>/acceptance-tested-revision.txt, diff the current worktree against that SHA, size the result by seams moved, and explore that. The coverage floor does not apply again. Do not re-run the previous pass, and do not derive the new scope from what was tested before.

If that file is missing, or names a SHA that does not resolve in this repository, report it as an impediment and explore the change as a whole rather than guessing a base.

Read the prior exploration log as history, meaning "this area already behaves this way," never as coverage, meaning "this area is signed off." An empty diff sizes to zero steps.

Wait for CI and hand off

Once no finding remains, require a clean tracked worktree, a pushed local HEAD, and a matching PR head before invoking codagent:wait-ci. If any of those is missing, tell the caller to commit, push, or align the PR; do not do it here. After CI returns, re-read local HEAD and the PR head and require both to match the returned SHA.

On failed, read the failing check's logs; on comments, read the comments. Then apply "Before blaming the environment" before reporting either as external. Add anything attributable to the implementation to the findings file and report it; report a confirmed external failure as a blocker with its log path and a short verbatim excerpt rather than a defect. On pending, report and stop. With no configured checks, continue after recording that CI coverage is absent. Treat skipped, neutral, cancelled, and timed-out checks as evidence states, never as passes.

Then write <evidence-directory>/acceptance-handoff.md:

  1. Decision brief: delivered behavior, unresolved decisions (read the ledger again now; never say none are pending without it showing none), overall status, suggested review path.
  2. Revision identity: repository, PR URL, current head SHA, tracked worktree status.
  3. Exploration: coverage-floor results, the budget and rationale, what was exercised, what was found, and the named list of what was not exercised. For each limitation attributed to a failed run, include the evidence path, a short verbatim excerpt of the failing message or warning, and the provenance line; give the reason for each item that could not be run.
  4. CI: status and durable links. Say plainly when no automated-validation evidence was supplied.
  5. Visual and client evidence: screenshot metadata, text equivalents, and practical review steps.
  6. Human-only obligations: applicable HT-* items as instructions, with no execution outcome.
  7. Assumptions: the ledger path, then each unresolved decision named by its ledger heading with its options and consequence, each material limitation the ledger, findings, or log record, and each risky or notable assumption in the implementation session report, if one was supplied. A count or a pointer alone does not carry a decision.

Before reporting ready, confirm every referenced path exists, that any scratch folder created by this skill was removed, and that no persisted evidence exposes secrets, credentials, or private data. Report the ready SHA, PR URL, handoff path, unresolved-decision count, CI status, and known limitations.

Out of scope

  • Do not fix defects, modify tracked source or approved artifacts, commit, push, or alter PR metadata. Disposable setup and build outputs are fine; outside the worktree, put them under the scratch folder.
  • Do not run automated suites, linters, or the Validator. Setup and builds needed to use the product are fine.
  • Do not change approved requirements, scope, or design to make something pass.
  • Do not obtain human acceptance, mark the PR ready, merge, archive, or release.
  • Do not fuzz or build an exhaustive edge-case matrix. Exploration is targeted by risk, not by volume.
  • Do not treat a successful command exit, an agent summary, or a screenshot alone as proof of behavior.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.