fixture-sync
Skill8 starsChanged 7 months agoApache-2.0View on GitHub
Five of eight current fixtures violate their schemas. These are documented and intentional — no runtime schema validation exists in agent.sh. Proceed only if you are intentionally fixing drift: For fields with enum constraints (particularly patterns.json type and structure-scout.json role): When fixing, use real values — never hand-craft values. Capture from real claude output if possible: Fixing a schema violation can break existing tests if a test was asserting on the (incorrect) current value. Update the assertion alongside the fixture.
In this repository22 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- .github/workflows/CLAUDE.mdCLAUDE.md
- lib/CLAUDE.mdCLAUDE.md
- prompts/CLAUDE.mdCLAUDE.md
- schemas/CLAUDE.mdCLAUDE.md
- scripts/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- add-bats-testSkill
- add-edge-testSkill
- add-gather-agentSkill
- add-integration-testSkill
- add-synthesis-fixtureSkill
- audit-prompt-schema-syncSkill
- bundle-and-verifySkill
- check-ci-locallySkill
- debug-agent-failureSkill
- pre-pr-checklistSkill
- rebuild-test-imageSkill
- release-ultrainitSkill
- run-testsSkill
- tune-synthesis-contextSkill
- validate-generated-artifactSkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 79
When fixing, use real values — never hand-craft values.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Before You Start
tests/fixtures/findings/— 8 fixture files, one per Phase 1 agentschemas/— 14 JSON schema files; the 8 relevant ones match by base nametests/fixtures/CLAUDE.md— documents the 5 known schema violations in current fixtures
Known Violations
Five of eight current fixtures violate their schemas. These are documented and intentional — no runtime schema validation exists in agent.sh. Proceed only if you are intentionally fixing drift:
| Fixture | Known Violation |
patterns.json | type: 'design' is not in the schema's enum |
tooling.json | Missing required purpose field |
git-forensics.json | Uses path where schema expects file; missing commit_patterns.example_messages |
structure-scout.json | Invalid role strings; extra total_files/total_directories fields |
Steps
1. Check which fixtures have required-field violations
for agent in identity commands git-forensics patterns tooling docs-scanner security-scan structure-scout; do
fixture="tests/fixtures/findings/${agent}.json"
schema="schemas/${agent}.json"
if [[ ! -f "$fixture" || ! -f "$schema" ]]; then
echo "MISSING: $agent"
continue
fi
# Check required fields
required=$(jq -r '.required[]?' "$schema" 2>/dev/null)
for field in $required; do
present=$(jq -e ".${field}" "$fixture" 2>/dev/null && echo yes || echo no)
if [[ "$present" == "no" ]]; then
echo "MISSING REQUIRED: $agent.$field"
fi
done
done2. Check for additionalProperties violations
for agent in identity commands git-forensics patterns tooling docs-scanner security-scan structure-scout; do
fixture="tests/fixtures/findings/${agent}.json"
schema="schemas/${agent}.json"
fixture_keys=$(jq -r 'keys[]' "$fixture" 2>/dev/null | sort)
schema_keys=$(jq -r '.properties | keys[]' "$schema" 2>/dev/null | sort)
extra=$(comm -23 <(echo "$fixture_keys") <(echo "$schema_keys"))
if [[ -n "$extra" ]]; then
echo "EXTRA FIELDS in $agent: $extra"
fi
done3. Check enum values
For fields with enum constraints (particularly patterns.json type and structure-scout.json role):
# Check patterns.json type values
jq -r '.patterns[].type' tests/fixtures/findings/patterns.json | sort -u
# Compare against schema enum:
jq -r '.properties.patterns.items.properties.type.enum[]' schemas/patterns.json | sort4. Fix a fixture
When fixing, use real values — never hand-craft values. Capture from real claude output if possible:
# Option A: Minimal fix — add missing required field
jq '.findings[0] += {"purpose": "example purpose"}' \
tests/fixtures/findings/tooling.json > /tmp/fixed.json
mv /tmp/fixed.json tests/fixtures/findings/tooling.json
# Option B: Replace invalid enum value
jq '.patterns[].type |= if . == "design" then "architecture" else . end' \
tests/fixtures/findings/patterns.json > /tmp/fixed.json
mv /tmp/fixed.json tests/fixtures/findings/patterns.json5. Verify tests still pass after fixing
make test-unit
make test-integrationFixing a schema violation can break existing tests if a test was asserting on the (incorrect) current value. Update the assertion alongside the fixture.
Verify
# All fixtures are valid JSON
for f in tests/fixtures/findings/*.json; do jq empty "$f" && echo "OK: $f"; done
# Run tests
make test-unitCommon Mistakes
- Hand-crafting fixture values — the
tests/fixtures/CLAUDE.mdrule says to capture realclaude -p --output-format jsonoutput. Hand-crafted fixtures can make tests pass against data that real Claude would never produce. - Fixing a fixture without updating dependent tests — some unit tests assert on specific fixture field values. If you fix
patterns.json's type enum, find any test that assertstype: 'design'and update it too.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

