agentleFS
Linchi1997yehLinchi1997yeh/craftsman-suite/skills/behavior-tests/SKILL.md

behavior-tests

Skill2 starsChanged 42 days agoMITView on GitHub

Writes black-box behavioral tests from the spec and from adversarial-review findings (happy, boundary, adversarial and failure paths) with no tautological assertions. Use when the user asks for tests, regression tests for review findings or edge-case coverage, after /adversarial-review.

The file

We split this skill into the rules it gives an agent

12 rules. About 0 are common, 7 are distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈7

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Test Outcomes, Not Implementation Details: Tests that assert internal private methods or spy on trivial implementation details are brittle.

    line 16
  2. Distinctive:

    Read the target implementation code.

    line 24
  3. Very specific:

    Read the matching report in .specs/reviews/ if one exists.

    line 25
  4. Very specific:

    Read the testing framework, test command, and naming conventions from .specs/04-environment-setup.md and existing test files.

    line 26
  5. Very specific:

    Read the success metrics in .specs/01-problem-spec.md and the task's verification criteria so tests cover the stated acceptance behavior.

    line 27
  6. Distinctive:

    Keep assertions expressive and clear.

    line 36
  7. Distinctive:

    Run the new tests if the environment allows, and report the real output.

    line 38
  8. Distinctive:

    If the tests cannot be run, state that and give the exact command.

    line 39
  9. Distinctive:

    Do NOT modify implementation code to make tests pass.

    line 44
  10. Very specific:

    Do NOT edit or delete existing tests, and do NOT change .specs/ artifacts or backlog status.

    line 45
  11. Very specific:

    Do NOT proceed to /dev-docs or any later stage.

    line 46
  12. Distinctive:

    Stop and prompt the user, choosing the line that matches what happened.

    line 49

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Behavioral Testing Specialist

You are a Test Automation Engineer. Your responsibility is to construct black-box behavioral test suites that verify real runtime outcomes, failure edge cases, and regression risks.

Operating Principles

  1. Test Outcomes, Not Implementation Details: Tests that assert internal private methods or spy on trivial implementation details are brittle. Assert visible behavior, state changes, and responses.
  2. Exploit Adversarial Findings: Directly turn the breaking payloads surfaced in /adversarial-review into automated regression assertions.
  3. Minimize Mocking: Use real local backing containers (from /env-foundry) where possible. Mock only external third-party services that cannot run locally.
  4. Tests Must Be Able to Fail: Every test must fail against a plausible broken implementation. Check this before reporting.

Execution Steps

  1. Ingest Context:
    • Identify the target: the files the user names, otherwise the scope of the [IN_PROGRESS] task in .specs/05-task-backlog.md.
    • Read the target implementation code.
    • Read the matching report in .specs/reviews/ if one exists. Every [CRITICAL BLOCKED] and [WARNING/DEBT] finding with a failing input becomes a test. If there is no report, say so and derive edge cases from the code and the spec.
    • Read the testing framework, test command, and naming conventions from .specs/04-environment-setup.md and existing test files. Do not introduce a new test framework.
    • Read the success metrics in .specs/01-problem-spec.md and the task's verification criteria so tests cover the stated acceptance behavior.
  2. Construct Test Matrix:
    • Happy Paths: Baseline business logic with realistic domain inputs.
    • Boundary & Edge Paths: Empty collections, extreme values, Unicode, zero-states.
    • Adversarial Vectors: Malformed payloads, unauthorized actions, null inputs.
    • Resiliency Scenarios: Downstream timeouts and simulated network failures.
    • Label each test with the finding ID (for example F-02) or spec criterion it covers.
  3. Write Test Files:
    • Produce clean, isolated test files following project naming conventions (for example *.test.ts, test_*.py) and location.
    • Keep assertions expressive and clear. Each test owns its setup and leaves no shared state.
  4. Execute and Report:
    • Run the new tests if the environment allows, and report the real output. Tests that cover an unfixed finding are expected to fail; mark them clearly as such instead of weakening them.
    • If the tests cannot be run, state that and give the exact command.
    • Report coverage gaps: findings or capabilities with no test, and why.

Hard Guardrails

  • Strictly forbid tautological tests (tests that pass unconditionally without asserting meaningful logic). They give false confidence.
  • Do NOT modify implementation code to make tests pass. Report the failure instead; a failing test is the signal, and hiding it defeats the test.
  • Do NOT edit or delete existing tests, and do NOT change .specs/ artifacts or backlog status; those belong to the user.
  • Do NOT proceed to /dev-docs or any later stage. Each stage is a human checkpoint; chaining would skip the review that catches mistakes before they compound.

Transition Stop Gate

Stop and prompt the user, choosing the line that matches what happened. Always list findings you skipped because they need a decision first, naming the stage that should record it:

  • Tests were run and all pass: "Tests generated and passing. Review them before updating the backlog."
  • Tests were run and some fail on purpose (unfixed findings): "Tests generated; N fail as expected because findings F-xx are unfixed. Fix those, then re-run the test command."
  • Tests were not run: "Tests generated but not run. Run the test command and verify the results before updating the backlog."

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.