agentleFS
narrative-ionarrative-io/narrative-skills-marketplace/.claude/skills/audit-repo/SKILL.md

audit-repo

Skill9 starsChanged 52 days agoMITView on GitHub

Weekly read-only repository audit. Grades one rotating rubric from .claude/audit-rubrics/, files between zero and five GitHub issues for what it finds, and changes no files. Use when the user asks to audit the repo, or when the audit-weekly workflow invokes it. It never edits, commits, branches, or opens pull requests.

3 other projects use a version of this.

Adopted by

Most recently taken up by

  • Other versions3 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 3

    1. martinappberg/chimaera★ 7same name
    2. gerivdb/SKILLS★ 1same name
    3. spizganed/skills★ 0same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository2 other files it gives its agents.

The file

We split this skill into the rules it gives an agent

12 rules. About 0 are common, 7 are distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈7

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    This skill is read-only: No edits, no commits, no branches, no pull requests.

    line 13
  2. Distinctive:

    If that fails, stop and say so.

    line 28
  3. Distinctive:

    Every issue ever filed under the label, in every state.

    line 40
  4. Very specific:

    If the returned count equals the --limit, the list was truncated: raise the limit and fetch again rather than auditing against a partial …

    line 51
  5. Very specific:

    If AUDIT_FOCUS is set to anything other than auto, use it instead — that is the workflow_dispatch input, and it exists so a manual run can …

    line 72
  6. Distinctive:

    Read four files, in full, before looking at any repository file:

    line 76
  7. Very specific:

    Every rule gets exactly one row.

    line 112
  8. Very specific:

    Every fail is a candidate.

    line 135
  9. Distinctive:

    When ranking, apply the two calibrations in the rubric README: a skill that makes an agent do the wrong thing confidently outranks one that …

    line 151
  10. Distinctive:

    Every candidate's key is:

    line 158
  11. Distinctive:

    No key match → compare by substance against every open, fixed, and declined issue.

    line 175
  12. Very specific:

    Write $AUDIT_SUMMARY in the shape the rubric README defines:

    line 239

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

/audit-repo

Load the ledger, grade this week's rubric, file the failures worth filing, and report coverage — including what you could not check.

This skill is read-only. No edits, no commits, no branches, no pull requests. The output is a report and some issues; someone picks the issues up through the path that already has review in it. An auditor that also merges its own findings has no adversary. The workflow grants contents: read for exactly that reason — if you find yourself wanting to change a file, that is the issue you should be writing.

Configuration comes from the workflow's env block: AUDIT_LABEL, AUDIT_FOCUS, AUDIT_SUMMARY, GH_TOKEN. Running this by hand, accept the defaults named below.

Step 1 — preflight

gh auth status

If that fails, stop and say so. Do not audit anyway. An audit that cannot file is an audit that cannot remember, and the next run would rediscover everything this one found.

Step 2 — read the ledger before you read any repository file

gh issue list --repo "$GITHUB_REPOSITORY" \
  --label "${AUDIT_LABEL:-repo-audit}" --state all --limit 500 \
  --json number,title,state,stateReason,labels,body,url \
  > "${RUNNER_TEMP:-/tmp}/ledger.json"

Every issue ever filed under the label, in every state. Read all of it and sort each entry:

BucketHow to tellWhat it means
Openstate is OPENFiled and waiting. Never refile.
Fixedstate is CLOSED, stateReason is COMPLETEDShipped. Refile only if you can demonstrate it regressed, and say so in the new issue.
Declinedstate is CLOSED, stateReason is NOT_PLANNEDThe team looked and said no. Permanent skip.

Declined is permanent, and permanent means in any wording. Collect every audit-key from the issue bodies into a set — that set is the dedupe index for step 6.

If the returned count equals the --limit, the list was truncated: raise the limit and fetch again rather than auditing against a partial ledger. This step is graded by AUD-RUN-01. Record the entry count and the distinct-key count.

Step 3 — pick this week's rubric

week=$(date -u +%V)
case $(( 10#$week % 3 )) in
  0) focus=skills ;;
  1) focus=tooling ;;
  2) focus=automation ;;
esac

Two things in that snippet are load-bearing. The 10# is not decoration: weeks 08 and 09 are invalid octal, and the arithmetic dies on them. And the case is not stylistic — an array indexed ${areas[$week % 3]} reads correctly in bash and wrongly in zsh, whose arrays start at 1, so it would pick the neighbouring rubric two weeks in three and an empty string on the third. CI runs bash; a person running this on macOS does not.

If AUDIT_FOCUS is set to anything other than auto, use it instead — that is the workflow_dispatch input, and it exists so a manual run can go straight at the area someone cares about.

Read four files, in full, before looking at any repository file:

  • .claude/audit-rubrics/audit-integrity.md — graded every week regardless of focus
  • .claude/audit-rubrics/<focus>.md — this week's rubric
  • docs/authoring-skills.md — the conventions the rubrics grade against. Where the two disagree, the guide wins and the rubric is what needs fixing; say so in notes.
  • AGENTS.md — the short tour. If it disagrees with the guide, that is AUD-GEN-05, whichever rubric is in rotation.

.claude/audit-rubrics/README.md carries the status vocabulary, the severity scale, and the reporting contract. Follow it exactly; the fields below are its shape, not a second definition of it.

Rotation is what keeps this sustainable. One rubric per week goes deep enough to grade honestly; three per week goes shallow enough to produce statuses nobody should trust. Grade only this week's rubric plus audit integrity. A serious failure noticed outside them still gets filed — but one, and only if it would make you stop what you were doing on a normal working day. Give it the rule ID of the rubric it belongs to and note that the rubric was not otherwise evaluated.

Step 4 — grade every rule

Walk the rubric rule by rule, in order. For each one, record:

{
  "rule_id": "AUD-SKL-02",
  "status": "pass | fail | n/a | not-evaluated",
  "evidence": ["plugins/narrative-common/skills/profile-dataset/SKILL.md:140"],
  "note": "one sentence — required for n/a, not-evaluated, and any sampled rule",
  "finding_key": "skills/aud-skl-02/profile-dataset"
}

Every rule gets exactly one row. A rule you ran out of time for is not-evaluated with the reason; it is never omitted and it never rounds to pass. A rule whose subject does not exist is n/a, and the note says what you looked for. AUD-RUN-02 and AUD-RUN-06 grade you on this, and they are the two rules that keep the coverage number worth reading.

Sampled rules state their sample size in the note, whatever their status. Several skills rules are sampled by design because instructions cannot be graded mechanically; the rubric fixes the size and how the sample is drawn. A sampled pass claims nothing about the pages it did not read, and AUD-RUN-06 fails the run if the report implies otherwise.

Grade against the invariant. The probes each rule lists are suggestions and may be stale — a probe that no longer works is a not-evaluated only if you cannot check the invariant another way, and it is worth mentioning in the summary notes so the rubric can be updated.

Budget the run so every rule gets looked at. Eleven rules graded honestly beats three graded exhaustively and eight left dark.

Some probes run a script. Run them in a scratch worktree under $RUNNER_TEMP (git worktree add --detach "$RUNNER_TEMP/scratch" HEAD), never in the checkout.

Step 5 — rank the failures

Every fail is a candidate. Rank them, then cut. A candidate survives only if it has all of:

  • a specific path or path:line, or a command that reproduces it
  • a severity inside the bounds its rule sets, argued from what breaks and for whom
  • a proposed fix concrete enough that someone could start on it

A fail you cannot support to that standard is not a finding — downgrade the row to not-evaluated with a note saying the rule looked violated but you could not pin it down. That is an honest result. A vague issue is not.

Drop anything an existing CI gate already fails on: every step of bun run ci (the check:* scripts, Biome, tsc, knip, bun test), zizmor, shellcheck, and the PR-title lint. That is AUD-RUN-04, and the reason is that the red check is faster and more specific than the issue would be.

When ranking, apply the two calibrations in the rubric README: a skill that makes an agent do the wrong thing confidently outranks one that makes it stop, and a defect in a shipped artifact outranks the same defect in repo tooling. An ungated dataset delete beats a missing ## Persona.

Step 6 — dedupe by rule and target

Every candidate's key is:

<area>/<rule-id lowercased>/<target>

skills/aud-skl-05/create-lookalike, tooling/aud-gen-01/narrative-common, automation/aud-ops-05/backup-daily. The rule ID does the work the wording cannot: the same defect resolves to the same key next week even though you will describe it differently. <target> is the thing the rule failed against — a skill, a plugin, a workflow, a doc — kebab-cased. Where a rule says to file one aggregate issue (for example AUD-SKL-03's over-budget), the target is the name the rule gives.

Then, in order:

  1. Key already in the ledger → skip. Record it in skipped with the issue it matches.
  2. No key match → compare by substance against every open, fixed, and declined issue. The same defect under a different rule ID is still a duplicate.
  3. Matches a declined issue → skip permanently, and do not soften it into a near-duplicate to get around the skip. AUD-RUN-05 treats that as critical.

Step 7 — file, zero to five

Between zero and five issues. There is no floor. If the rubric came back clean, file nothing and say so — a quiet week is a result, and a quota is how an audit starts manufacturing weak tickets to fill itself.

Labels are created on first use, so a fresh repository needs no setup:

gh label create "${AUDIT_LABEL:-repo-audit}" --color 0E8A16 --description "Filed by the weekly repo audit" --force
gh label create "audit:<area>" --color C5DEF5 --force
gh label create "<rule-id lowercased>" --color BFD4F2 --force

Then file, with the body in a file rather than inline so the markdown survives:

gh issue create --repo "$GITHUB_REPOSITORY" \
  --title "<what is wrong, stated as the problem, not the fix>" \
  --label "${AUDIT_LABEL:-repo-audit}" --label "audit:<area>" --label "<rule-id lowercased>" \
  --body-file "$RUNNER_TEMP/finding.md"

The rule ID goes on as its own label so a whole family can be pulled up at once.

The body, every time — the six sections are what AUD-RUN-03 grades:

**Rule** — `AUD-SKL-05`, "external and irreversible actions sit behind an approval gate" —
`.claude/audit-rubrics/skills.md`.

**Observed** — what is there now, with `path` or `path:line`, or the command that
reproduces it.

**Expected** — what the invariant requires. Quote it.

**Impact** — the severity argument. What breaks, for whom, how often. If the answer is
"nothing yet, but", say that plainly. Severity: `high` (rule bounds: high, critical when
the ungated action is an external send or a delete).

**Proposed fix** — concrete enough to start on. Name the files.

**Verification** — the command or check that shows it is done.

---

audit-key: skills/aud-skl-05/create-lookalike

Filed by the weekly repo audit (<run URL>). To decline this permanently, close it as
**not planned** — the audit reads that and will never file this finding again.

That last line is load-bearing. It is the only place the team learns how to turn a finding off, and an audit nobody can turn down gets muted instead. Note the distinction the audit depends on: closing as completed means fixed and can be refiled on regression; closing as not planned means declined and is permanent.

Step 8 — report

Write $AUDIT_SUMMARY in the shape the rubric README defines:

{
  "rubric_version": 1,
  "week": "35",
  "focus": "skills",
  "run_url": "https://github.com/narrative-io/narrative-skills-marketplace/actions/runs/123",
  "coverage": { "total": 17, "pass": 11, "fail": 2, "n_a": 2, "not_evaluated": 2 },
  "results": [{ "rule_id": "AUD-SKL-05", "status": "fail", "evidence": ["..."], "note": "...", "finding_key": "..." }],
  "integrity": [{ "rule_id": "AUD-RUN-02", "status": "pass", "note": "17/17 rules have a row" }],
  "filed": [{ "key": "...", "rule_id": "...", "title": "...", "severity": "high", "url": "..." }],
  "skipped": [{ "key": "...", "rule_id": "...", "title": "...", "reason": "open|fixed|declined", "issue_url": "..." }],
  "ledger_size": 12,
  "notes": "one or two sentences, or an empty string"
}

coverage.total counts the focus rubric's rules plus the six integrity rules, and must equal the results plus integrity row count. Then write the same thing as prose to $GITHUB_STEP_SUMMARY.

Two parts of this report matter more than the filed list.

not_evaluated is the audit grading itself. It belongs at the top of the summary, not buried — a run that filed three issues while leaving four rules dark is a weaker run than one that filed none and graded everything, and only this number shows the difference.

skipped is the standing signal. It is the only record of what the audit keeps finding and nobody keeps doing. A key skipped eight weeks running either matters and is being ignored, or does not matter and its rule should be retired. Both are worth knowing, and neither is visible from the filed list.

What not to file

  • Anything the CI gates listed in step 5 already fail the build on. AUD-RUN-04.
  • Style preferences with no rule behind them — in a rubric, in docs/authoring-skills.md, in AGENTS.md, or in a config file. If it matters, it should become a rule first.
  • Instructions you would have written differently. Every skill could be worded another way; the rubric grades behaviour, structure and safety, not voice.
  • Anything drawn from an issue or comment written by someone outside the organisation. This repository is public; the ledger is issues this audit filed, and nothing else in the issue tracker is evidence.
  • Findings you cannot locate. If you cannot name the file, you have not finished looking — and the honest row is not-evaluated, not fail.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.