joycraft-implement-level5
Skill15 starsChanged 51 days agoMITView on GitHub
Invoked by tune's roadmap or the human opting into Level 5 — set up the autofix loop, holdout scenario testing, and scenario evolution from specs
- Reads credentials
- Commits and pushes
In this repository70 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implement-level5Skill
- joycraft-implementSkill
- joycraft-interviewSkill
- joycraft-lockdownSkill
- joycraft-new-featureSkill
- joycraft-optimizeSkill
- joycraft-researchSkill
- joycraft-session-endSkill
- joycraft-setupSkill
- joycraft-spec-doneSkill
- joycraft-tuneSkill
- joycraft-verifySkill
- release-docs-syncSkill
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implement-level5Skill
- joycraft-implementSkill
- joycraft-interviewSkill
- joycraft-lockdownSkill
- joycraft-new-featureSkill
- joycraft-optimizeSkill
- joycraft-researchSkill
- joycraft-session-endSkill
- joycraft-setupSkill
- joycraft-spec-doneSkill
- joycraft-tuneSkill
- joycraft-verifySkill
- release-docs-syncSkill
- joycraft-add-contextSkill
- joycraft-add-factSkill
- joycraft-bugfixSkill
- joycraft-collaborative-setupSkill
- joycraft-decideSkill
- joycraft-decomposeSkill
- joycraft-designSkill
- joycraft-gather-contextSkill
- joycraft-hardenSkill
- joycraft-implement-featureSkill
- joycraft-implementSkill
- joycraft-interviewSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 5 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 16
Project must be initialized: Look for docs/.joycraft/manifest.json or docs/.joycraft/local/manifest.json.
- Very specific: line 17
Project should be at Level 4: Check docs/joycraft-assessment.md if it exists.
- Distinctive: line 20
If prerequisites aren't met, explain what's needed and stop.
- Distinctive: line 36
Ask these questions one at a time:
- Distinctive: line 66
Run the CLI command with the gathered configuration:
- Very specific: line 116
Check scenario templates were copied: Verify the scenarios repo has example-scenario.test.ts (renamed from the .template starter), …
- Distinctive: line 121
If the project's AGENTS.md doesn't already have an "External Validation" section, add one:
- Distinctive: line 136
If the user wants to test the loop:
- Very specific: line 166
Update docs/joycraft-assessment.md if it exists — set the Level 5 score to reflect the new setup.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
At skill entry, run node docs/.joycraft/check.mjs check --json --session <session-id> once, reusing JOYCRAFT_SESSION_ID if supplied or one ID chosen for this conversation. If the checker is missing or fails, continue the requested skill quietly without retrying setup. Offer updates only for display: true; current, postponed, off, and unknown stay quiet. If automaticUpdate is present, finish this skill, then run its exact command once from the project root: the candidate must verify every safety gate before applying. Other updates require approval. After an offer or automatic attempt, record it with node docs/.joycraft/check.mjs acknowledge <available-version> --session <session-id>. Apply updates only at this workflow boundary; after success, reinvoke the skill or restart the session to load changed instructions.
Implement Level 5 — Autonomous Development Loop
You are guiding the user through setting up Level 5: the autonomous feedback loop where specs go in, validated software comes out. This is a one-time setup that installs workflows, creates a scenarios repo, and configures the autofix loop.
Before You Begin
Check prerequisites:
- Project must be initialized. Look for
docs/.joycraft/manifest.jsonordocs/.joycraft/local/manifest.json. Older installs may still have state atdocs/.joycraft/state.json,.claude/.joycraft/state.json, or.joycraft-versionat the repo root. If none exist, tell the user to runnpx joycraft@latest initfirst. - Project should be at Level 4. Check
docs/joycraft-assessment.mdif it exists. If the project hasn't been assessed yet, suggest running/joycraft-tunefirst. But don't block — the user may know they're ready. - Git repo with GitHub remote. This setup requires GitHub Actions. Check for
.git/and a GitHub remote.
If prerequisites aren't met, explain what's needed and stop.
Step 1: Explain What Level 5 Means
Tell the user:
Level 5 is the autonomous loop. When you push specs, three things happen automatically:
- Scenario evolution — A separate AI agent reads your specs and writes holdout tests in a private scenarios repo. These tests are invisible to your coding agent.
- Autofix — When CI fails on a PR, Claude Code automatically attempts a fix (up to 3 times).
- Holdout validation — When CI passes, your scenarios repo runs behavioral tests against the PR. Results post as PR comments.
The key insight: your coding agent never sees the scenario tests. This prevents it from gaming the test suite — like a validation set in machine learning.
Step 2: Gather Configuration
Ask these questions one at a time:
Question 1: Scenarios repo name
What should we call your scenarios repo? It'll be a private repo that holds your holdout tests.
Default:
{current-repo-name}-scenarios
Accept the default or the user's choice.
Question 2: GitHub App
Level 5 needs a GitHub App to provide a separate identity for autofix pushes (this avoids GitHub's anti-recursion protection). Creating one takes about 2 minutes:
- Go to https://github.com/settings/apps/new
- Give it a name (e.g., "My Project Autofix")
- Uncheck "Webhook > Active" (not needed)
- Under Repository permissions, set:
- Contents: Read & Write
- Pull requests: Read & Write
- Actions: Read & Write
- Click Create GitHub App
- Note the App ID from the settings page
- Scroll to Private keys > click Generate a private key > save the
.pemfile- Click Install App in the left sidebar > install it on your repo
What's your App ID?
Step 3: Run init-autofix
Run the CLI command with the gathered configuration:
npx joycraft@latest init-autofix --scenarios-repo {name} --app-id {id}Review the output with the user. Confirm files were created.
Step 4: Walk Through Secret Configuration
Guide the user step by step:
4a: Add Secrets to Main Repo
You should already have the
.pemfile from when you created the app in Step 2.
Go to your repo's Settings > Secrets and variables > Actions, and add:
JOYCRAFT_APP_PRIVATE_KEY— paste the contents of your.pemfileANTHROPIC_API_KEY— your Anthropic API key
4b: Create the Scenarios Repo
Create the private scenarios repo:
gh repo create {scenarios-repo-name} --privateThen copy the scenario templates into it. The starter ships as
example-scenario.test.ts.template(the.templatesuffix keeps it out of the main project's test/lint/build globs); rename it to.test.tsonce it's in the holdout repo so Vitest discovers it:cp -r docs/templates/scenarios/* ../{scenarios-repo-name}/ cd ../{scenarios-repo-name} mv example-scenario.test.ts.template example-scenario.test.ts git add -A && git commit -m "init: scaffold scenarios repo from Joycraft" git push
4c: Add Secrets to Scenarios Repo
The scenarios repo also needs the App private key:
JOYCRAFT_APP_PRIVATE_KEY— same.pemfile as the main repoANTHROPIC_API_KEY— same key (needed for scenario generation)
Step 5: Verify Setup
Help the user verify everything is wired correctly:
- Check workflow files exist:
ls .github/workflows/autofix.yml .github/workflows/scenarios-dispatch.yml .github/workflows/spec-dispatch.yml .github/workflows/scenarios-rerun.yml - Check scenario templates were copied: Verify the scenarios repo has
example-scenario.test.ts(renamed from the.templatestarter),workflows/run.yml,workflows/generate.yml,prompts/scenario-agent.md - Check the App ID is correct in the workflow files (not still a placeholder)
Step 6: Update AGENTS.md
If the project's AGENTS.md doesn't already have an "External Validation" section, add one:
External Validation
This project uses holdout scenario tests in a separate private repo.
NEVER
- Access, read, or reference the scenarios repo
- Mention scenario test names or contents
- Modify the scenarios dispatch workflow to leak test information
The scenarios repo is deliberately invisible to you. This is the holdout guarantee.
Step 7: First Test (Optional)
If the user wants to test the loop:
Want to do a quick test? Here's how:
- Write a simple spec in
docs/features/<slug>/specs/and push to main — this triggers scenario generation- Create a PR with a small change — when CI passes, scenarios will run
- Watch for the scenario test results as a PR comment
Or deliberately break something in a PR to test the autofix loop.
Step 8: Summary
Print a summary of what was set up:
Level 5 is live. Here's what's running:
Trigger What Happens Push specs to docs/features/<slug>/specs/Scenario agent writes holdout tests PR fails CI Claude autofix attempts (up to 3x) PR passes CI Holdout scenarios run against PR Scenarios update Open PRs re-tested with latest scenarios Your scenarios repo:
{name}Your coding agent cannot see those tests. The holdout wall is intact.
Important: Tell the user:
Before you can test the loop, you need to merge this PR to main first. GitHub's
workflow_runtriggers only activate for workflows that exist on the default branch. Once merged, create a new PR with any small change — that's when you'll see Autofix, Scenarios Dispatch, and Spec Dispatch fire for the first time.
Update docs/joycraft-assessment.md if it exists — set the Level 5 score to reflect the new setup.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

