agentleFS
hsigstadhsigstad/research-kit/skills/check/SKILL.md

check

Skill0 starsChanged 4 months agoMITView on GitHub

Audit a project or pipeline against workspace conventions AND propose fixes for everything found. Three modes: docs (doc-contract + source/build naming), ledgers (artifacts.yaml + validation.yaml + .run.json + cross-refs), cite (citation tokens against the registry + anchors + .bib). All three are fast and deterministic. Default runs all three and proposes changes. Use when the user invokes /check, /check <mode>, /check <slug>, or asks to 'lint the docs', 'audit conventions', 'check citations', 'check the ledgers', 'clean up the docs'.

  • Installs packages

80 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use daily-arxiv, exp-eval and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • LambenthanSkill

    daily-arxiv

    每日从 arXiv 拉取新论文,过滤相关性,auto-ingest 高优先级论文,检测 SOTA 更新

    Also in 5 projects that use this

  • LambenthanSkill

    exp-eval

    实验判决门:Review LLM 独立评判实验结果 → 4 种判决路径 → 自动更新 claims confidence、ideas status、graph edges

    Also in 5 projects that use this

  • JuliusBrusseeSkill

    backprop

    Bug → spec protocol. When a bug is found or a test fails, trace the cause, decide whether a new §V invariant would catch recurrence, append to §B. This is the one non-obvious thing SDD does that plan-then-execute doesn't. Triggers on test failure, bug report, post-mortem, or explicit user ask.

    Also in 4 projects that use this

  • skyllwtSkill

    exp-pilot-eval

    Pilot result evaluation — read pilot results, apply success criteria, update idea page (pilot_result, failure_reason if failed), generate PILOT_VERDICT_REPORT. Called by /ideate Phase 5 after /exp-pilot-run.

    Also in 4 projects that use this

  • skyllwtSkill

    exp-pilot-run

    Pilot experiment execution — read Pilot Spec YAML, write pilot code, run experiment(Confirm with the user before operation and require the applicant to conduct manual inspection), return results. Called by /ideate Phase 5. Does NOT modify wiki pages or judge pass/fail.

    Also in 4 projects that use this

  • +6 more from Lambenthan
  • +1 more from skyllwt

The file

We split this skill into the rules it gives an agent

3 rules. 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • 0

    Common

    Most of the 69 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If any tool errors, surface the traceback — do not silently fall back.

    line 75
  2. Distinctive:

    Use the underlying tool's grouped output for all three modes.

    line 88
  3. Very specific:

    Don't wrap routed skills in plan mode: When a ledger gap routes to /findings --update, let that child skill handle its own approval — …

    line 180

“Common” is measured: kept by at least 6 in 10 of the 69 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

/check — Workspace convention audit (with proposed fixes)

Orchestrates the three fast deterministic project audits, proposes fixes for every finding, and lets the user approve before any change lands. Thin wrapper over the underlying tools — the skill's job is sequence, unified reporting, and the propose-then-apply interaction shape.

The default behavior is propose-and-apply, not read-only. Pass --json when you want a raw report for tooling or CI.

ModeWhat it auditsBacking
docsdoc-contract presence, todo/done hygiene, thinking.md sections, source/build naming (incl. multi-output folder), pandas merge validate=, archive.md leakage, idea frontmatterresearch-kit/tools/check_docs.py (deterministic, <2s workspace-wide)
ledgersartifacts.yaml ↔ disk drift, validation.yaml pending/stale rows, .run.json sidecar provenance, doc ↔ artifacts.yaml cross-refsresearch-kit/tools/coverage.py (deterministic; requires pyyaml)
cite[ns:key] citation tokens against the workspace registry, internal anchors, and .bib filesresearch-kit/tools/citations.py (deterministic; lint only — no manifest sync)

For IAT (INTENT / REASONING / ASSUMES / SOURCE) comment compliance, see /iat — that audit is LLM-driven and slower, kept as its own skill so /check stays fast.

For manifest regeneration (docs/refs/manifest.toml), see /cite-sync — that's a write action, not an audit.

How to invoke

FormWhat it does
/checkAll three modes for the current project (inferred from cwd) or workspace-wide; propose fixes and ask before applying
/check <slug>All three modes against a single project/pipeline
/check docsDocs mode only
/check ledgersLedgers mode only
/check citeCitation mode only
/check <mode> <slug>One mode, one repo
/check --fullShow every instance in the report (not the grouped summary)
/check --jsonRaw JSON output, no proposals — for CI, tooling, or "just tell me how dirty it is"

Finding the workspace root

Workspace root contains CLAUDE.md alongside projects/, pipelines/, ideas/, research/. Search upward from cwd; project root is $ROOT/projects/<slug>/. If neither cwd nor an explicit slug resolves, fail with a useful message.

Procedure

1. Resolve mode and scope

  • Mode: docs, ledgers, cite, or all (default). If the first positional arg matches a mode keyword, treat it as the mode; otherwise treat it as a slug and use mode all.
  • Scope: explicit slug, cwd's project, or workspace-wide if neither.

2. Run the underlying tool(s)

For each requested mode:

# docs
python3 $ROOT/research-kit/tools/check_docs.py [<slug>] --json

# ledgers (project-scoped only — no workspace-wide rollup)
python3 $ROOT/research-kit/tools/coverage.py --project $PROJ --json

# cite (lint mode; --sync is /cite-sync's responsibility)
python3 $ROOT/research-kit/tools/citations.py [<slug>] --json

If any tool errors, surface the traceback — do not silently fall back. If coverage.py is missing, surface a hint to clone research-kit. If pyyaml is missing, tell the user to pip install pyyaml; don't try to install it yourself.

3. Render the focused report

Open with a one-line summary: N gaps across {docs, ledgers, cite} and the totals.

For each mode, group findings by code and show counts + first 3 examples, truncating with … +N more. --full enumerates every instance.

Use the underlying tool's grouped output for all three modes.

4. Propose fixes

This is the default. For every finding, derive a concrete fix (or a routed follow-up). The interaction shape differs by mode because the work differs:

Docs (/check docs): enter plan mode with the full fix list.

Each finding gets a templated fix from fixes.md (sibling file). Build the complete plan as a numbered list grouped by repo, then call EnterPlanMode with the plan. The user sees every change at once and approves or redirects before any edit lands. On approval, execute the file ops (renames, moves, new files, merge validate= insertions) and commit per repo with the message format in fixes.md. Findings without a templated fix (.merge() validate= value choice, archive.leakage) go in a Manual review section at the end of the plan — surfaced but not auto-applied. Common fixes:

GapDefault fix
doc.missingCreate the file with the canonical heading template
doc.unallowedRename to canonical (e.g. todos.md → todo.md) or move to briefs//notes//reference/ based on content
*.naming-violationmkdir the script-named folder and git mv the sibling outputs into it
*.orphan / *.orphan-folderEither rename the script to match, or delete the orphan (ask which)
source.no-output (data layers)No fix — surface as manual ("build may be unbuilt")
source.merge.no-validateManual review — pick one_to_one, many_to_one, etc. per call
project.no-claude-mdCreate CLAUDE.md at project root with a one-line summary + pointer to docs/summary.md
gitignore.no-buildAdd build/ to .gitignore (create the file if missing)
decisions.bad-headerRewrite the header to ## YYYY-MM-DD — <title> (em-dash). Auto-fix if the date is parseable; manual otherwise
handoff.bad-filenamegit mv to <ISO-timestamp>_<short-tag>.md (manual: pick the right tag)
source.underscore-citedManual review — either rename the script to drop the underscore (and let it be a real artifact) or rewrite the citing doc to point elsewhere
analysis.design.unknown-keyManual review — either fold the value into an allowed design: key (or notes), or, if it recurs across analyses, add the key to docs/reference/analysis-schema.yaml
stale.runtime-disclaimerAuto — delete the NOT runtime-tested / parse-checked only line. The affordance was retired 2026-05-29 (see research/rules/synthetic_and_bypasses.md); the disclaimer is residual text, not a live convention. /check flags residue only; the queue/export skill (e.g. /connect-run) gates new scripts on the actual runtime test

Ledgers (/check ledgers): numbered list, per-gap apply/skip/quit.

Plan mode is a bad fit here because most ledger gaps route to other gated skills (/findings --update, /hypothesis --extend, /validate-section). Wrapping those in plan mode means double-gating. Instead, present a numbered list of suggested follow-ups; the user picks which to invoke, and each child skill runs its own approval flow. Common routes:

GapRouted follow-up
entries with missing artifact on diskFix the path field (or delete the entry) — surface diff
entries with missing producing scriptFix the script field
entries with empty cited_in (load-bearing)/findings --extend
build/{table,figure} files not indexedAppend to artifacts.yaml
scripts missing IAT/iat <script>
pending validation rows older than 60d/validate-section
stale validation rows (hash drift)/validate-section
sidecars with commit_dirty: trueCommit and re-run, or document the dirty state

Cite (/check cite): enter plan mode for auto-fixable; manual review for the rest.

Mix of deterministic and judgment-call findings. Build the plan from the deterministic ones; surface the others in a Manual review section.

CodeAuto / manualDefault proposal
cite.malformedAutoDeterministic transform: lowercase ns, kebab-case key. Show old → new diff per occurrence
cite.prose-styleManualProse author-year citation (e.g. Smith (2020)) — replace with [cite:<bibkey>]. Surface each occurrence with the surrounding line; user picks the bibkey (or adds a bib entry if missing). High-false-positive risk for paper drafts that paraphrase quoted titles; treat as a soft prompt, not a blocker
cite.orphan-manifestAutoRun /cite-sync <slug> — regenerates the manifest from the live cited set, so the orphan entry drops out atomically (don't hand-edit the TOML)
cite.missing-from-manifestAutoRun /cite-sync <slug> — adds the missing entry by regenerating from the live cited set. Group multiple cite.missing-from-manifest + cite.orphan-manifest findings for the same project into a single /cite-sync invocation
cite.unresolved-externalManualSurface the missing [ns:key]; ask the user to add a registry entry (title, description, optional path)
cite.unresolved-internalManualShow the expected anchor (- id: ns:key or \label{ns:key}) and point at the doc where the citation lives
cite.unresolved-bibManualShow the [cite:key] and ask for a .bib stub
cite.unknown-namespaceManualEither a typo (suggest closest valid one) or a new namespace proposal — surface for review
cite.registry-danglingManualThe registry entry's path no longer exists. Ask the user to update or remove
anchor.duplicateManualTwo anchors with the same ID in the same scope (docs/ globally, or a single .tex file). Ask the user which to rename — show both occurrences

For manifest regeneration use /cite-sync (write action, not part of the audit flow).

5. Closing summary

Always end with:

  • N gaps surfaced; K approved & applied; L skipped or routed; M flagged as manual review.
  • Per-repo commit summary (if docs-mode applied any fixes).
  • Recommended next step if any ✗ items remain.
  • The raw commands so the user can re-run for a passive view:
    • python3 research-kit/tools/check_docs.py [<slug>] --json
    • python3 research-kit/tools/coverage.py --project <path> --json
    • python3 research-kit/tools/citations.py [<slug>] --json

Guardrails

  • Always propose, never auto-apply. --json is the only mode that skips proposals; even there, exit on a non-zero error count for CI gating.
  • Use plan mode for docs and the auto-fixable cite findings. A unified plan beats per-gap walking — the user sees the whole shape and can redirect in one step.
  • Don't wrap routed skills in plan mode. When a ledger gap routes to /findings --update, let that child skill handle its own approval — chaining plan-mode invocations confuses the gate.
  • Don't invoke /next from here. /check is a static audit. If a gap reads like new analysis work, surface it as a /next candidate hint, but don't start /next.
  • Don't try to fix things outside the listed gaps. Scope is bounded.
  • For IAT, route to /iat. It's kept separate so /check stays fast.

When to run

  • Before a /next session — clears drift so propagation in step 5 doesn't compound it.
  • After a parser/data refresh — sidecar dirty flags, broken artifact paths, and stale validation rows surface together.
  • After a script rename or directory move — broken artifact/script paths and naming violations fire immediately.
  • Weekly — the signal degrades cleanly when nothing's wrong, so a no-news run is a feature, not a problem.

Common failure modes

  • coverage.py not found. The project may not have research-kit cloned as a sibling. Tell the user to clone it.
  • pyyaml missing. Tell the user to pip install pyyaml (or use the project's virtualenv).
  • Project has no CLAUDE.md or docs//source/ directories. Auto-detect fails. Pass <slug> explicitly, or surface that the project isn't structured per the workspace conventions.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.