harness-audit
Skill0 starsChanged 3 days agoMITView on GitHub
Audit AGENTS.md, CLAUDE.md and linked project instructions against repository code, scripts, CI and agent configuration. Use to find stale commands, broken references, conflicting rules or missing verification paths; setup-project owns initial 2build configuration.
- Reads credentials
26 other projects use a version of this.
Adopted by
Most recently taken up by
- hoshiori-dev/harness.mdtheir own version, first seen Nov 16, 2026
- lumlaolao/spike-evaltheir own version, first seen Nov 16, 2026
- gutohuida/AgentWeavetheir own version, first seen Nov 15, 2026
- 5uck1ess/devkittheir own version, first seen Nov 15, 2026
- venhdev/skillstheir own version, first seen Nov 8, 2026
- DaeSeokSong/oh-my-ouroborostheir own version, first seen Nov 1, 2026
Used with
Projects with it also use harness-design, project-init and 3 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
VictoriakaeySkillharness-design
Use when designing or improving the action space, observation format, tool boundaries, and evaluation signals of an LLM agent. Covers the five layers of harness design: what tools the agent has, what it sees after acting, how each tool is scoped, how behavior is evaluated, and how to iterate when the agent misbehaves. Prevents the most common harness failures — agents that pick wrong tools, ignore critical information, or loop without progress.
Also in 3 projects that use this
u9401066Skillproject-init
Initialize new projects using this template. Triggers: init, new, 新專案, 初始化, create project, 建立專案, bootstrap, scaffold project.
Also in 3 projects that use this
nicolasestremSkillbrainstorm
Structured idea generation and solution exploration with systematic analysis.
Also in 3 projects that use this
MicrosoftSkillhandoff
Compact the current conversation into a handoff document for another agent to pick up.
Also in 3 projects that use this
GoogleSkilltdd
Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
Also in 3 projects that use this
Other versions26 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 26
- AnastasiyaW/claude-code-config★ 214same name
- AnastasiyaW/codex-claude-code-config★ 214same name
- Kanevry/session-orchestrator★ 53same name
- fabianmartinelli-fm/harness-audit★ 7same name
- fmslutions/harness-audit★ 7same name
- fernandopetry/keelson★ 5same name
- 5uck1ess/devkit★ 4same name
- gutohuida/AgentWeave★ 4same name
- DaeSeokSong/oh-my-ouroboros★ 2same name
- lumlaolao/spike-eval★ 2same name
16 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository30 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- llms.txtllms.txt
- agent-first-docsSkill
- analytics-reviewSkill
- architectSkill
- autopilotSkill
- browseSkill
- create-prSkill
- design-uiSkill
- foremanSkill
- humanizer-viSkill
- image-genSkill
- implementSkill
- investigateSkill
- marketing-videoSkill
- perf-auditSkill
- plan-draftSkill
- qaSkill
- reconSkill
- release-prdSkill
- releaseSkill
- review-prSkill
- semantic-decisionSkill
- setup-projectSkill
- teamSkill
- telemetry-reviewSkill
- testSkill
- token-auditSkill
- triageSkill
- ui-auditSkill
The file
We split this skill into the rules it gives an agent
10 rules. 0 are common, 4 are distinctive, and 6 are very specific to this project.
0
Common
Most of the 22 edited versions we read keep these.
4
Distinctive
This author's own choices. Worth a look.
6
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- could be automatic: a setting could enforce it
- Distinctive: line 7
Check whether an agent can discover the right constraints, run the project and verify a change from the instructions actually available to …
- Distinctive: line 11
Follow preamble for bootstrap and telemetry, and Auto-Decision Framework for decisions.
- Distinctive: line 21
Do not create branches, commit, push, deploy, install tools, rotate credentials or change global settings as part of an audit.
- Very specific: line 24
Start at the requested repo/service or file.
- Very specific: line 30
Follow links needed to establish an operational claim.
- Very specific: line 78
Read git flow only for 2build git checks.
- Very specific: line 84
When --local release support is claimed, trace its checks, builds, packaging, publication and deploy commands statically against CI and …
- Distinctive: line 102
Check file existence, anchors and script/target definitions first.
- Very specific: line 110
Do not eval resolver output, source .env, or run commands copied from docs just to audit them.
- Very specific: line 134
If fixes are authorized, patch the owning file, preserve unrelated user edits and existing policy, then recheck affected …
“Common” is measured: kept by at least 6 in 10 of the 22 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
harness-audit
Check whether an agent can discover the right constraints, run the project and verify a change from the instructions actually available to it. Findings need repository evidence, not a preferred documentation template.
Follow preamble for bootstrap and telemetry, and
Auto-Decision Framework for decisions.
Shared refs are filesystem paths beside this skill's directory, so read them
by path, not as skill://.
Scope and authority
Audit is read-only by default, apart from normal skill telemetry and a report when requested. An explicit request to fix or optimize the harness authorizes scoped edits; do not turn “check AGENTS.md” into setup, policy changes or a rewrite of every linked doc. No ticket or 2build configuration is required. Do not create branches, commit, push, deploy, install tools, rotate credentials or change global settings as part of an audit.
Start at the requested repo/service or file. Discover root and relevant nested
AGENTS.md / CLAUDE.md, imports/symlinks and referenced architecture, design,
run/test/deploy/release docs. Also search each service for an unlinked
architecture document (ARCHITECT*.md, ARCHITECTURE*.md, docs/architecture*.md);
an existing but unlinked one is a reachability finding. Discover the linked release profile, otherwise
.babysit/release.md, RELEASE.md or an existing inline release section.
Follow links needed to establish an operational claim;
exclude dependency/vendor/generated trees unless referenced. Do not traverse
every sibling repo or entire monorepo for a service-scoped request.
Record which instructions apply to each checked service, including inherited and local rules. Use the actual harness's loading/override rules when known; do not assume Codex, Claude Code and other hosts load the same files. If that is unknown, report the assumption. Missing one filename is not itself a defect when another entrypoint correctly serves the intended harness.
Evidence checks
First check the project context links
contract: AGENTS.md must lead to the project's architecture guidance, and to
design/deployment/release guidance when those apply. Verify service coverage and when
each document should be read. Distinguish a missing link to existing guidance
from missing documentation; report either gap even if there is no file to
inspect. Do not require a particular filename or design/deployment docs for
projects that do not need them. Apply the owner audit in
project docs:
per-service architecture coverage, a system document limited to cross-service
scope, and every remaining TODO(2build): placeholder listed as an open gap.
For a releasable app, CLI, library or pack, use
the release skill's profile contract
to audit release coverage. An equivalent linked inline section/runbook is
valid; missing RELEASE.md alone is not a finding. Missing release guidance
for an actual release surface, an unreachable existing profile or conflicting
duplicate owners is a finding. A non-releasable repo can mark this N/A.
The profile must say where each release unit deploys, how, and what is
verified and observed after deploy; a field the repo contradicts is drift
that the next release also corrects, so name the field and its evidence.
Then inspect the following where relevant or present:
| Surface | Compare against | Report when |
| Instruction entrypoints and linked docs | Actual paths, link anchors, imports/symlink targets and service ownership | Broken links, circular imports, unreachable required guidance or instructions applying to the wrong service |
| Run/build/test commands | Package scripts, lockfiles, Makefile targets, compose, runner config and CI; resolve working directory and prerequisites | Command missing, wrong runtime/port/cwd, suites excluded from documented checks, required infrastructure omitted |
| Architecture and operational rules | Representative implementation and current configuration at the named path | Guidance contradicts reality, duplicates have drifted, ownership or exceptions are ambiguous |
| Architecture owners | Service directories, nested entrypoints and the system/service split in project docs | A service without an owner; system doc not linking a service doc; system doc holding service internals, schema detail, deploy env/commands, content conventions, roadmaps, resolved history or account IDs that another owner holds (name the target owner) |
| Placeholders | rg -n 'TODO\(2build\):' across entrypoints and linked owners | Each remaining placeholder (file:line, missing fact) as an open gap; a placeholder the repository already answers is drift |
| Base branch | git flow resolution against actual refs | Resolves to a branch the repo does not release from; a leftover .babysit/git-flow.yaml is ignored (note, not defect) |
.babysit/qa.yaml and local overrides | Effective local environment, service commands, credential variable names and meaningful flows | Wrong/missing target, hosted-only QA, no failure/empty/validation case, override changes target or credential source unexpectedly |
Release profile (RELEASE.md, .babysit/release.md or linked inline owner) | Version/mirror/lockfile sources, tags/changelog, CI/script stage ownership, package registries, current deploy/rollback runbook, observation rules and CLI installation paths | Wrong repo/component/version identity, stale commands, missing required stage or unjustified N/A, ambiguous artifact/SHA, unverifiable hypercare bounds, missing local update verification or profile unreachable from entrypoints |
| Ignore rules and examples | Tracked filenames, .gitignore, .env.example, configuration references | Secret-bearing local files tracked or not ignored, committed credential literals, names pointing at the wrong environment |
| Browser automation hygiene | Browser/QA bullets in the entrypoint against /bbs:browse § Browser hygiene; scripts, CI or docs that launch Chrome | Entrypoint tells agents to use the installed Chrome, pkill/kill/close --all as cleanup, or an ad-hoc session name; a browser-using project with no close-the-browser instruction |
| Related repo/workspace pointers | Only relevant entries in ~/.babysit/config.yaml, named local paths and RELATED_*_REPO fallback | Referenced repo unavailable, roles disagree, machine paths committed, project-local .babysit/config.yaml mistaken for the registry |
Read git flow only for 2build git checks.
Missing .babysit files or workspace registration in a repo that does not use
those features is not a finding. For CLI/library repos, validate their actual
check path instead of requiring a browser URL. An unavailable submodule or
machine-local repo is an evidence gap, not proof its documentation is wrong.
When --local release support is claimed, trace its checks, builds, packaging,
publication and deploy commands statically against CI and helpers. Report
missing CI-equivalent checks, remote build/Actions dependencies disguised as
local execution, absent guard/restoration for relevant automatic workflows,
or instructions that bypass landing/authentication policy. Check that original
workflow states survive failures/resumes and already published artifacts are
verified before retry. Distinguish an intentional unsupported local path from
a false claim of support; missing live permissions are unverified facts.
For instruction quality, keep project-specific constraints and failure lessons that change decisions. Flag generic filler, repeated policy, stale examples or large always-loaded detail only when they obscure a real task or drift from an owner. Recommend one owning file plus precise pointers; do not delete rules merely to hit a line/token budget. If implementation violates a deliberate invariant, report that conflict rather than rewriting the invariant to match broken code. Separate factual drift from policy decisions that need an owner.
Validate without executing the instructions blindly
Check file existence, anchors and script/target definitions first. Parse YAML with an available parser; CLI output is additional evidence, not a complete schema validator. Inspect these read-only resolvers when 2build is configured:
bbs secrets qa default-env
bbs secrets qa probe --env <local-name>Choose the actual local name from config; a simple top-level target is local.
Do not eval resolver output, source .env, or run commands copied from docs
just to audit them. Inspect tracked/ignored status without dumping secrets.
Capture and redact command output before reporting if URLs or legacy configs
contain credentials. Record file/key and risk, never secret values.
Resolve target/cwd/command consistency statically by default. A safe read-only
probe of an already running local service can strengthen evidence; report it
separately from configured intent. Execute application checks or start services
only when runtime verification is requested, after inspecting side effects and
prerequisites. Missing tools/services become NOT RUN with a reason. Never run
migration, rollback, seed, deployment or destructive prepare hooks to “verify”
documentation. Auditing release guidance also does not authorize version bumps,
tags, publication, workflow suspension/dispatch or local CLI upgrades, including
npm pack --dry-run whose lifecycle hooks can mutate/build. A health response
proves reachability, not auth or user flows;
use browse / qa for requested UI execution.
Findings, optional fixes and output
For each actionable finding give severity, file:line, the conflicting evidence, impact on a concrete agent task, and the smallest correction. Prioritize unsafe commands/secret exposure, instructions that prevent execution or verification, then drift and maintainability. Separate confirmed defects, suggestions and unverified claims. Do not invent a numerical harness score.
If fixes are authorized, patch the owning file, preserve unrelated user edits
and existing policy, then recheck affected links/config/commands. Reuse
setup-project when the requested fix is missing
2build configuration or a missing architecture/release owner (skeleton with
TODO(2build): placeholders). Moving over-scoped content to its owner is a
fix only when requested; otherwise report it. Do not silently switch release model, credential
names or command semantics as a documentation cleanup.
Return the report directly unless the caller requests a file or supplies a handoff destination. Include inspected scope, prioritized findings, checks performed, changes made and gaps. Findings do not prevent completing an audit; inability to inspect its primary target does.
STATUS: DONE | DONE_WITH_CONCERNS | NEEDS_CONTEXT | BLOCKED
VERDICT: CLEAN | FINDINGS | INCOMPLETE
SCOPE: <entrypoints/services and linked files actually inspected>
VERIFY: <static checks, optional runtime evidence, NOT RUN and why>
GAPS: <TODO(2build) count per file, or none>
NEXT: <smallest correction, missing evidence, or none>CLEAN covers only the inspected scope and requires no unresolved material
verification gaps. INCOMPLETE means required evidence is missing; it is never
an application QA PASS. Report known findings even when the audit is incomplete.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

