harness-generation
Skill4 starsChanged 3 months agoMITView on GitHub
Seed minimal agent harness v0 — manifest, eval checks stub, governance. AUTO-INVOKED after project-setup or retroactive-project-setup when docs/harness/manifest.json is missing. Also triggers on: generate harness, scaffold agents, agent bootstrap, first time agents in this repo, new project agent setup, set up agent harness, agent onboarding files, missing agent configuration, agent instructions setup, make agents read project rules, agent reliability setup, agents not configured. Pairs with project-setup. Evolution is harness-evolution.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- specula-org/Speculatheir own version, first seen Oct 8, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- specula-org/Specula★ 636same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository125 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .cursor/rules/agent-loom-routing.mdcCursor rule
- .cursor/rules/agent-loom-skills-index.mdcCursor rule
- adversarial-hatSkill
- agent-builderSkill
- agent-launcherSkill
- agent-loom-syncSkill
- agent-observabilitySkill
- agent-run-retroSkill
- agent-system-architectureSkill
- api-and-interface-designSkill
- api-deprecation-and-migrationSkill
- apply-paper-to-projectSkill
- app-security-hardeningSkill
- architectural-decision-logSkill
- assumption-mappingSkill
- brainstormingSkill
- browser-testing-with-devtoolsSkill
- business-modelingSkill
- ci-cd-and-automationSkill
- codebase-understandingSkill
- code-review-crspSkill
- code-simplificationSkill
- compress-skillSkill
- context-engineeringSkill
- create-agent-promptSkill
- cross-link-skillsSkill
- customer-discoverySkill
- debug-and-fixSkill
- deep-thinkingSkill
- dependency-mappingSkill
- deploy-anywhereSkill
- deprecate-skillSkill
- design-directionSkill
- design-reviewSkill
- design-systemSkill
- dynamic-routingSkill
- eval-judgeSkill
- eval-outputSkill
- eval-pipelineSkill
- eval-rubric-designSkill
- experimentationSkill
- experiment-backlogSkill
- experiment-readoutSkill
- experiment-runbookSkill
- experiment-specSkill
- fault-localizeSkill
- feature-specSkill
- fermiSkill
- first-principlesSkill
- frontend-designSkill
- generate-changelogSkill
- git-workflow-and-versioningSkill
- gsap-animationSkill
- harness-engineeringSkill
- harness-evolutionSkill
- idea-evaluationSkill
- idea-generationSkill
- implementation-planSkill
- improve-skillsSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 4 are distinctive, and 8 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
8
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 34
Auto-invoke: when chained from project-setup 6c, retroactive-project-setup, harness-engineering Step 0, or setup-evaluation harness FAIL.
- Very specific: line 38
Never generate a harness without a versioned component manifest (docs/harness/manifest.json).
- Distinctive: line 39
Never ship a pre-tuned bloated harness — v0 is minimal.
- Distinctive: line 40
Never rely on prompt-only harness — tools, middleware, skills, and eval interface are required surfaces (AHE ablation).
- Very specific: line 41
Never overwrite user-edited harness files without manifest drift check — refuse or merge explicitly.
- Very specific: line 42
Never claim self-improving without eval interface stub + pointer to harness-evolution.
- Distinctive: line 43
Never embed external repo URLs in generated artifacts — distill patterns locally.
- Very specific: line 60
Read: manifests, existing AGENTS.md, .agents/skills/, CI configs, docs/memory/, docs/evals/, docs/harness/.
- Very specific: line 65
Write docs/harness/manifest.json per references/component-manifest.md: seven orthogonal components (prompt, tools, middleware, skills, …
- Very specific: line 81
If project-setup already ran: merge, never replace user interview content.
- Very specific: line 85
Document harness sync --check equivalent: compare manifest hashes to disk.
- Distinctive: line 164
No eval interface stub when evolution is planned
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Harness Generation
You seed a minimal, auditable harness v0 — the scaffold around a model (prompts,
tools, skills, memory hooks, lifecycle, verification interface, governance). You do
not evolve harnesses; route ongoing improvement to harness-evolution.
Trigger Discipline
Auto-invoke when chained from project-setup 6c, retroactive-project-setup, harness-engineering Step 0, or setup-evaluation harness FAIL. See harness-engineering/references/harness-readiness-gate.md.
Hard Rules
Never generate a harness without a versioned component manifest (docs/harness/manifest.json).
Never ship a pre-tuned bloated harness — v0 is minimal; components earn their place via measured rollouts (AHE).
Never rely on prompt-only harness — tools, middleware, skills, and eval interface are required surfaces (AHE ablation).
Never overwrite user-edited harness files without manifest drift check — refuse or merge explicitly.
Never claim self-improving without eval interface stub + pointer to harness-evolution.
Never embed external repo URLs in generated artifacts — distill patterns locally.
Workflow
Step 1 — Classify delivery context
| Context | v0 focus |
| agent-loom skill library project | Skill routing, AGENTS.md orchestration map, eval stub for skill quality |
| Consumer app repo | AGENTS.md, .agents/skills/, forbidden paths, key commands |
| Legacy repo (no AGENTS.md) | Gap-fill only — pair with retroactive-project-setup first |
| Multi-agent chain planned | Manifest only — topology is agent-builder, not this skill |
Step 2 — Inventory repo (silent)
Read: manifests, existing AGENTS.md, .agents/skills/, CI configs, docs/memory/,
docs/evals/, docs/harness/. Auto-extract commands per project-setup Step 1b.
Step 3 — Emit component manifest
Write docs/harness/manifest.json per references/component-manifest.md:
seven orthogonal components (prompt, tools, middleware, skills, sub-agents, memory, governance),
each with path, version, sha256, and generated|user-edited status.
Step 4 — Scaffold artifacts
Apply patterns from references/scaffold-patterns.md:
- Prompt / routing — merge into
AGENTS.mdor create harness section; keep ≤150 lines total. - Skills — ensure Skill Invocation block present; list installed skills in Orchestration Map.
- Tools — document MCP/tool surfaces in manifest; stub
docs/harness/tools.mdif needed. - Governance — forbidden paths, allowed-write scopes, verifier sandbox rules.
- Eval interface stub —
docs/harness/eval-interface.md+ emptydocs/harness/tasks.jsonshape (file_phrase + command checks per metaharness pattern). - Environment bootstrap — compact snapshot block (stack, key commands, git branch) for meta-agents.
If project-setup already ran: merge, never replace user interview content.
Step 5 — Drift gate
Document harness sync --check equivalent: compare manifest hashes to disk; flag user-edited files
skipped on re-run. CI recommendation in manifest notes.
Step 6 — Hand off
| Next need | Route |
| Full AGENTS.md interview missing | project-setup |
| Eval rubric + regression suite | eval-rubric-design → eval-pipeline |
| Improve harness from failures | harness-evolution (requires eval harness) |
| Multi-agent design | agent-builder (orthogonal) |
Append to docs/skill-outputs/SKILL-OUTPUTS.md. Log harness version v0 in docs/memory/current-state.md if memory suite installed.
Gotchas
- Minimal seed is mandatory. Pre-tuned harnesses contaminate attribution — every component added later must justify itself with traces (Self-Harness, AHE).
- AGENTS.md must stay lean. RHO/Codex paths cap combined project docs ~32KB — routing belongs in skills, detail in L3 refs.
- Harness ≠ agent topology.
agent-builderdesigns who talks to whom; this skill designs what files/tools/prompts wrap the model. - Deterministic scaffold beats one-shot LLM init. Inspect → plan → staged writes → manifest (harnessforge pattern); LLM fill is optional fallback only.
- Blueprint validators that shell out are a trust boundary — document, never auto-run untrusted commands.
Output Format
Harness generation complete
Version: v0
Context: [library | consumer | legacy-gap-fill]
Manifest: docs/harness/manifest.json
Artifacts: [list paths]
Eval stub: docs/harness/eval-interface.md
Drift check: [documented command]
Next: [eval-pipeline | harness-evolution | project-setup]Example
<examples>
<example>
<input>Generate a harness for this React monorepo — we have AGENTS.md from project-setup.</input>
<output>
Harness generation complete
Version: v0 | Context: consumer
Manifest: docs/harness/manifest.json (7 components, all generated)
Merged: Skill Invocation + Orchestration Map preserved from interview
Created: docs/harness/eval-interface.md, docs/harness/tasks.json (stub), docs/harness/governance.md
Next: eval-rubric-design to define harness regression dimensions
</output>
</example>
</examples>Common Rationalizations
| Excuse | Reality |
| "AGENTS.md is enough" | No manifest, eval stub, or governance = not a harness v0 |
| "Skip eval stub for now" | harness-evolution hard-fails without it |
| "Copy external harness README" | Distill patterns; secure-scan; no URL embed |
| "Let LLM write the whole harness" | Deterministic manifest + merge beats opaque generation |
| "Agent-builder already ran" | Topology ≠ harness files — run both |
Verification
docs/harness/manifest.jsonexists with 7 components and hashes- Eval interface stub + tasks.json shape present
- AGENTS.md ≤150 lines; user content preserved on merge
- Governance forbidden paths documented
- SKILL-OUTPUTS.md appended
Red Flags
- Harness v0 bloated with speculative prompts/tools
- User-edited files overwritten without merge
- No eval interface stub when evolution is planned
- External SKILL.md vendored wholesale
Prune Log
Last pruned: 2026-07-05
- Deep learn-from: scaffold-patterns L3 — drift CI, interface validation, AHE sandbox
Impact Report
Harness v0 generated: [context]
Manifest: docs/harness/manifest.json | Components: 7
Eval stub: [yes/no] | Merged AGENTS.md: [yes/no]
Files created: [count] | Next route: [skill]Reference Files
references/component-manifest.md— seven components, JSON schema, ETCLOVG mapreferences/scaffold-patterns.md— inspect→render, drift CI, bootstrap snapshotreferences/examples.md— library project + consumer repo worked examples
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

