joycraft-verify
Skill15 starsChanged 54 days agoMITView on GitHub
Invoked by session-end or the human directly — spawn an independent verifier subagent to check an implementation against its spec, read-only
In this repository70 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implement-level5Skill
- joycraft-implementSkill
- joycraft-interviewSkill
- joycraft-lockdownSkill
- joycraft-new-featureSkill
- joycraft-optimizeSkill
- joycraft-researchSkill
- joycraft-session-endSkill
- joycraft-setupSkill
- joycraft-spec-doneSkill
- joycraft-tuneSkill
- release-docs-syncSkill
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implement-level5Skill
- joycraft-implementSkill
- joycraft-interviewSkill
- joycraft-lockdownSkill
- joycraft-new-featureSkill
- joycraft-optimizeSkill
- joycraft-researchSkill
- joycraft-session-endSkill
- joycraft-setupSkill
- joycraft-spec-doneSkill
- joycraft-tuneSkill
- joycraft-verifySkill
- release-docs-syncSkill
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implement-level5Skill
- joycraft-implementSkill
- joycraft-interviewSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 4 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 16
If the user provided a spec path (e.g., $joycraft-verify docs/features/<slug>/specs/add-widget.md), use that path directly.
- Very specific: line 18
If no path was provided, scan docs/features/*/specs/ recursively for spec files (and docs/bugfixes/<area>/ for bugfixes).
- Distinctive: line 24
Read the spec file and extract:
- Very specific: line 39
If no parent brief exists (a standalone spec), skip items 1-2 and note "standalone spec, no brief oracle" — AGENTS.md boundaries still …
- Distinctive: line 41
If the spec has no Acceptance Criteria section, tell the user:
- Distinctive: line 45
If the spec has no Test Plan section, note this but proceed -- the verifier can still check criteria by reading code and running any …
- Distinctive: line 49
Look for test commands in these locations (in priority order):
- Very specific: line 129
Write this verdict to the style contract in docs/templates/reference/output-style.md.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
At skill entry, run node docs/.joycraft/check.mjs check --json --session <session-id> once, reusing JOYCRAFT_SESSION_ID if supplied or one ID chosen for this conversation. If the checker is missing or fails, continue the requested skill quietly without retrying setup. Offer updates only for display: true; current, postponed, off, and unknown stay quiet. If automaticUpdate is present, finish this skill, then run its exact command once from the project root: the candidate must verify every safety gate before applying. Other updates require approval. After an offer or automatic attempt, record it with node docs/.joycraft/check.mjs acknowledge <available-version> --session <session-id>. Apply updates only at this workflow boundary; after success, reinvoke the skill or restart the session to load changed instructions.
Verify Implementation Against Spec
The user wants independent verification of an implementation. Your job is to find the relevant spec, extract its acceptance criteria and test plan, then spawn a separate verifier subagent that checks each criterion and produces a structured verdict.
Why a separate subagent? Research found that agents reliably skew positive when grading their own work. Separating the agent doing the work from the agent judging it consistently outperforms self-evaluation. The verifier gets a clean context window with no implementation bias.
Step 1: Find the Spec
If the user provided a spec path (e.g., $joycraft-verify docs/features/<slug>/specs/add-widget.md), use that path directly.
If no path was provided, scan docs/features/*/specs/ recursively for spec files (and docs/bugfixes/<area>/ for bugfixes). Pick the most recently modified .md file. If no specs exist, tell the user:
No specs found under
docs/features/*/specs/ordocs/bugfixes/. Please provide a spec path:$joycraft-verify path/to/spec.md
Step 2: Read and Parse the Spec
Read the spec file and extract:
- Spec name -- from the H1 title
- Acceptance Criteria -- the checklist under the
## Acceptance Criteriasection - Test Plan -- the table under the
## Test Plansection, including any test commands - Constraints -- the
## Constraintssection if present
Step 2.5: Gather Oracle Inputs Beyond the Spec (PROTOCOL)
The spec alone is not the oracle — a spec can silently drift from the brief it was decomposed from, and a verifier that only reads the spec would rubber-stamp that drift. Before spawning the verifier subagent, also gather:
- The parent brief's Hard Constraints -- read
docs/features/<slug>/brief.md(linked from the spec's> **Parent Brief:**line) and extract its Hard Constraints / non-negotiables section. - The brief's
decisions:frontmatter block -- the stamped decisions (id, question, status, choice, rationale) that constrained this feature. - AGENTS.md boundaries -- the project's ALWAYS / ASK FIRST / NEVER boundaries.
If no parent brief exists (a standalone spec), skip items 1-2 and note "standalone spec, no brief oracle" — AGENTS.md boundaries still apply. Missing decisions: frontmatter (legacy brief) is not an error — proceed without it.
If the spec has no Acceptance Criteria section, tell the user:
This spec doesn't have an Acceptance Criteria section. Verification needs criteria to check against. Add acceptance criteria to the spec and try again.
If the spec has no Test Plan section, note this but proceed -- the verifier can still check criteria by reading code and running any available project tests.
Step 3: Identify Test Commands
Look for test commands in these locations (in priority order):
- The spec's Test Plan section (look for commands in backticks or "Type" column entries like "unit", "integration", "e2e", "build")
- The project's AGENTS.md (look for test/build commands in the Development Workflow section)
- Common defaults based on the project type:
- Node.js:
npm testorpnpm test --run - Python:
pytest - Rust:
cargo test - Go:
go test ./...
- Node.js:
Build a list of specific commands the verifier should run.
Step 4: Spawn the Verifier Subagent
Spawn a concurrent subagent thread with the following prompt. Replace the placeholders with the actual content extracted in Steps 2-3.
Important: The subagent must be given read-only constraints. It may search the codebase, read files, and run the specified test/build commands, but it must NOT edit or create any files.
You are a QA verifier. Your job is to independently verify an implementation against its spec. You have NO context about how the implementation was done -- you are checking it fresh.
RULES -- these are hard constraints, not suggestions:
- You may search the codebase and read any file
- You may RUN these specific test/build commands: [TEST_COMMANDS]
- You may NOT edit, create, or delete any files
- You may NOT run commands that modify state (no git commit, no npm install, no file writes)
- You may NOT install packages or access the network
- Report what you OBSERVE, not what you expect or hope
SPEC NAME: [SPEC_NAME]
ACCEPTANCE CRITERIA:
[ACCEPTANCE_CRITERIA]
TEST PLAN:
[TEST_PLAN]
CONSTRAINTS:
[CONSTRAINTS_OR_NONE]
BRIEF HARD CONSTRAINTS:
[BRIEF_HARD_CONSTRAINTS_OR_STANDALONE]
BRIEF DECISIONS (decisions: frontmatter):
[BRIEF_DECISIONS_OR_NONE]
PROJECT BOUNDARIES (AGENTS.md):
[PROJECT_BOUNDARIES]
YOUR TASK:
Your oracle is the brief's Hard Constraints + decisions + project boundaries, not the spec in isolation — the spec is one implementer's translation of that oracle, and it can drift. For each acceptance criterion, determine if it PASSES or FAILS based on evidence:
1. Run the test commands listed above. Record the output.
2. For each acceptance criterion:
a. Check if there is a corresponding test and whether it passes
b. If no test exists, read the relevant source files to verify the criterion is met
c. If the criterion cannot be verified by reading code or running tests, mark it MANUAL CHECK NEEDED
3. For criteria about build/test passing, actually run the commands and report results.
4. Separately, compare the SPEC (its Constraints and Acceptance Criteria) against the BRIEF HARD CONSTRAINTS, BRIEF DECISIONS, and PROJECT BOUNDARIES above. If the spec contradicts, narrows, or omits something the brief/decisions/boundaries required, that is spec-vs-brief drift — report it as a FINDING, not folded silently into a criterion's pass/fail. A finding here is about the *spec*, not the implementation: don't auto-fail the implementation for a drift that originated upstream in the spec.
OUTPUT FORMAT -- you MUST use this exact format:
VERIFICATION REPORT
| # | Criterion | Verdict | Evidence |
|---|-----------|---------|----------|
| 1 | [criterion text] | PASS/FAIL/MANUAL CHECK NEEDED | [what you observed] |
| 2 | [criterion text] | PASS/FAIL/MANUAL CHECK NEEDED | [what you observed] |
[continue for all criteria]
FINDINGS (spec-vs-brief drift, if any):
- [finding text, or "none found"]
SUMMARY: X/Y criteria passed. [Z failures need attention. / All criteria verified.]
If any test commands fail to run (missing dependencies, wrong command, etc.), report the error as evidence for a FAIL verdict on the relevant criterion.Step 5: Format and Present the Verdict
Write this verdict to the style contract in docs/templates/reference/output-style.md.
Take the subagent's response and present it to the user in this format:
## Verification Report -- [Spec Name]
| # | Criterion | Verdict | Evidence |
|---|-----------|---------|----------|
| 1 | ... | PASS | ... |
| 2 | ... | FAIL | ... |
**Findings (spec-vs-brief drift):** [list, or "none found"]
**Overall: X/Y criteria passed.**
[If all passed:]
All criteria verified. Ready to commit and open a PR.
[If any failed:]
N failures need attention. Review the evidence above and fix before proceeding.
[If any MANUAL CHECK NEEDED:]
N criteria need manual verification -- they can't be checked by reading code or running tests alone.
[If any drift findings:]
Spec-vs-brief drift found -- this is a finding against the spec, not the implementation. Review before the spec is trusted as the oracle for future verification passes.Step 6: Suggest Next Steps
Based on the verdict:
- All PASS: Suggest committing and opening a PR, or running
$joycraft-session-endto capture discoveries. - Some FAIL: List the failed criteria and suggest the user fix them, then run
$joycraft-verifyagain. - MANUAL CHECK NEEDED items: Explain what needs human eyes and why automation couldn't verify it.
Do NOT offer to fix failures yourself. The verifier reports; the human (or implementation agent in a separate turn) decides what to do. This separation is the whole point.
Edge Cases
| Scenario | Behavior |
| Spec has no Test Plan | Warn that verification is weaker without a test plan, but proceed by checking criteria through code reading and any available project-level tests |
| All tests pass but a criterion is not testable | Mark as MANUAL CHECK NEEDED with explanation |
| Subagent can't run tests (missing deps) | Report the error as FAIL evidence |
| No specs found and no path given | Tell user to provide a spec path or create a spec first |
| Spec status is "Complete" | Still run verification -- "Complete" means the implementer thinks it's done, verification confirms |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

