agentleFS
jiahao-shao1jiahao-shao1/sjh-skills/skills/context-audit/SKILL.md

context-audit

Skill43 starsChanged 7 months agoMITView on GitHub

Audit project context management (CLAUDE.md, rules, knowledge) for progressive disclosure compliance. Checks that all knowledge files are reachable from rules, detects stale references, orphaned files, and CLAUDE.md index leakage. Triggers: 'audit context', 'check context', 'context hygiene', 'context audit', '检查 context', '审计上下文', 'knowledge 覆盖检查'. NOT for: code review, rule content correctness, or knowledge content quality.

  • Reads credentials

19 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use skillify and refactor.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • MicrosoftSkill

    skillify

    Capture a repeatable process from the current session into a reusable Amplifier SKILL.md skill file. Use when the user wants to create a skill, save a workflow as a skill, turn a process into a reusable skill, or mentions "skillify", "create skill", "make a skill", "save as skill", "capture workflow", "turn this into a skill", "new skill", or wants to automate a repeatable process they just performed.

    Also in 3 projects that use this

  • GitHubSkill

    refactor

    Surgical code refactoring to improve maintainability without changing behavior. Covers extracting functions, renaming variables, breaking down god functions, improving type safety, eliminating code smells, and applying design patterns. Less drastic than repo-rebuilder; use for gradual improvements.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

12 rules. 0 are common, 10 are distinctive, and 2 are very specific to this project.

  • 0

    Common

    Most of the 19 edited versions we read keep these.

  • 10

    Distinctive

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    After adding new knowledge files — verify they're indexed from a rule

    line 19
  2. Distinctive:

    After refactoring rules — verify no references broke

    line 20
  3. Distinctive:

    After onboarding a new domain — verify the new rule covers its knowledge files

    line 22
  4. Very specific:

    If docs/knowledge/ or .claude/rules/ doesn't exist, report and skip the corresponding checks.

    line 41
  5. Distinctive:

    Orphaned: not reachable from any rule, even transitively (action needed)

    line 53
  6. Distinctive:

    Verify target exists: check the referenced knowledge file actually exists on disk

    line 58
  7. Distinctive:

    Check that CLAUDE.md follows the architecture principle — it should describe the progressive disclosure system, NOT duplicate the detailed …

    line 68
  8. Distinctive:

    Check architecture description: look for keywords indicating the architecture is described (e.g., "渐进式披露", "progressive disclosure", …

    line 71
  9. Distinctive:

    If orphaned files are found, suggest concrete edits:

    line 132
  10. Distinctive:

    If audit reveals rules are too large (always-loaded context bloat), recommend the three-tier split approach (validated on agentic_umm …

    line 142
  11. Very specific:

    Create a sibling knowledge file (e.g., <rule-name>-howto.md, <rule-name>-details.md) and replace the moved sections with a one-line > 详见 …

    line 156
  12. Distinctive:

    If no facts are supplied and no recent commits exist, skip silently rather than fabricate

    line 234

“Common” is measured: kept by at least 6 in 10 of the 19 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Context Audit

Audits the project's three-layer context management system (CLAUDE.md / rules / knowledge) against the progressive disclosure architecture:

CLAUDE.md (always loaded) — describes architecture, no detailed index
  └─ Rules (.claude/rules/) (always loaded) — inline `> 详见` refs to knowledge
       └─ Knowledge (docs/knowledge/) — on-demand deep experience

When to Use

  • After adding new knowledge files — verify they're indexed from a rule
  • After refactoring rules — verify no references broke
  • Periodic hygiene check — catch drift over time
  • After onboarding a new domain — verify the new rule covers its knowledge files

Execution

Phase 1: Discovery

Scan the project to find all context files:

# Knowledge files
find docs/knowledge/ -name "*.md" -type f 2>/dev/null | sort

# Rule files
find .claude/rules/ -name "*.md" -type f 2>/dev/null | sort

# CLAUDE.md
ls CLAUDE.md 2>/dev/null

If docs/knowledge/ or .claude/rules/ doesn't exist, report and skip the corresponding checks.

Phase 2: Cross-Reference Audit

For each knowledge file found in Phase 1:

  1. Grep rule files first for references to that knowledge file (match docs/knowledge/<filename> or bare <filename>)
  2. If not found in rules, grep knowledge index files (docs/knowledge/*-index.md or files whose name ends with -index.md). An index file that is itself rule-referenced transitively covers the leaves it lists.
  3. Record which rule(s) and/or index reference it
  4. Mark files as:
    • Directly covered: referenced from at least one rule
    • Indirectly covered: referenced from a knowledge index that is itself rule-covered (legal two-tier pattern)
    • Orphaned: not reachable from any rule, even transitively (action needed)
    • Deprecated: contains [DEPRECATED] marker (info only, no action needed)

For each reference found in rule/AGENTS/CLAUDE files:

  1. Verify target exists: check the referenced knowledge file actually exists on disk
  2. Skip placeholder examples: refs that look like syntax illustrations (e.g., docs/knowledge/xxx.md, <filename>.md, foo.md) inside example/format sections are not real refs. Detect by:
    • filename matches xxx, foo, bar, <.*>, or
    • the surrounding line uses template-like wording (e.g., 内联 详见 docs/knowledge/xxx.md 引用)
  3. Mark real references as:
    • Valid: target file exists
    • Stale: target file doesn't exist (action needed)

Phase 3: CLAUDE.md Compliance

Check that CLAUDE.md follows the architecture principle — it should describe the progressive disclosure system, NOT duplicate the detailed index.

  1. Count direct knowledge references: grep CLAUDE.md for docs/knowledge/ patterns
  2. Check architecture description: look for keywords indicating the architecture is described (e.g., "渐进式披露", "progressive disclosure", "rules", "详见")
  3. Flag if CLAUDE.md contains more than 3 direct knowledge file references (suggests index leakage — the detailed index belongs in rules, not CLAUDE.md)

Exception: a summary table mapping rule files to knowledge domains is acceptable (it's architecture description, not a detailed index).

Phase 4: Report

Output a structured report:

==========================================
  CONTEXT AUDIT REPORT
  <project-name> | YYYY-MM-DD
==========================================

## Coverage Summary

Knowledge files: N total
  - M directly covered (cited from a rule)
  - K indirectly covered (cited from a rule-linked knowledge index)
  - X orphaned
  - Y deprecated
Rule files: N total
Stale references: N (placeholders excluded)

## Orphaned Knowledge Files (not indexed from any rule)

| File | Suggested Rule | Reason |
|------|---------------|--------|
| docs/knowledge/foo.md | rl-training.md | topic overlaps with RL training |

## Stale References (pointing to non-existent files)

| Rule File | Line | Reference | Status |
|-----------|------|-----------|--------|
| .claude/rules/bar.md | 42 | docs/knowledge/removed.md | file not found |

## CLAUDE.md Compliance

- Architecture description: [FOUND / MISSING]
- Direct knowledge refs: N (threshold: <=3)
- Status: [PASS / WARN — index leakage detected]

## Coverage Map

| Rule File | Knowledge Files Referenced |
|-----------|--------------------------|
| rl-training.md | reward, config, verl-integration, ... |
| cluster-ops.md | container-unify, mutagen-sync, ... |

## Deprecated Files

| File | Deprecation Notice |
|------|--------------------|
| docs/knowledge/experiments.md | migrated to experiment-registry |

==========================================

Phase 5: Fix Suggestions (optional)

If orphaned files are found, suggest concrete edits:

  • Identify the most relevant rule file based on topic overlap
  • Suggest the specific section where a > 详见 reference should be added
  • Provide the exact line to add (copy-pasteable)

Do NOT auto-apply fixes. Present suggestions and let the user decide.

Phase 6: Rule Slimming Methodology (when rules are bloated)

If audit reveals rules are too large (always-loaded context bloat), recommend the three-tier split approach (validated on agentic_umm 2026-04-25, reduced 8 rules / ~830 lines → 6 rules / 469 lines, -43%):

Tier 1 — Whole-file migration (highest ROI)

Identify rules that are pure how-to / reference manuals with no hard constraints. Move the entire file to docs/knowledge/, update the index pointer in CLAUDE.md/AGENTS.md to point at knowledge instead.

Signals: rule reads like a tutorial, no "必须 / 不要 / 硬约束" language, pure command recipes or organization conventions.

Tier 2 — In-file split (medium ROI)

For mixed rules, split into:

  • Keep in rule: hard constraints (project-specific invariants where violation causes incidents — data corruption, eval mismatch, lost work)
  • Move to knowledge: how-to procedures, detailed diagnostic stories, reference tables, command templates

Create a sibling knowledge file (e.g., <rule-name>-howto.md, <rule-name>-details.md) and replace the moved sections with a one-line > 详见 docs/knowledge/<...>.md.

Tier 3 — Index extraction (low-medium ROI)

If a rule contains a long flat table indexing many knowledge files (e.g., 20+ rows like "数据/轨迹/评估"), extract that table into a dedicated docs/knowledge/<topic>-index.md. The rule keeps a single sentence pointing to the index. The index file is itself rule-referenced, so leaves remain reachable (legal two-tier pattern recognized in Phase 2).

Decision rules:

  • Hard constraint → rule (always-loaded)
  • How-to / reference / diagnostic detail → knowledge (on-demand)
  • Long flat index of knowledge files → dedicated index knowledge file
  • Cross-tool agents (e.g., Codex) that don't load .claude/rules/ → may need direct knowledge refs in AGENTS.md; this is acceptable cross-tool-compat redundancy, not a violation

Phase 7: Change Impact Matrix (forward audit, opt-in)

Phases 1-6 are reverse audits — start from existing files, check structural compliance. Phase 7 is the forward audit — start from recent changes, check whether the knowledge they introduced actually landed in every place it should.

This phase is opt-in: trigger it when the user supplies a change summary, or when the audit follows a sizable session (feature shipped, refactor done, bug fixed). Skip silently if there's nothing to forward-audit.

Inputs

Collect "new facts" introduced by recent work. Sources, in order of preference:

  1. User-supplied change summary (most reliable)
  2. git log --since="<session start>" --oneline + git diff of touched files
  3. Recent conversation context (least reliable — confirm with user before acting on it)

A "new fact" is anything that, if the next session reads stale docs, would mislead the agent. Examples: dependency swap (SQLite → PostgreSQL), new env var, renamed CLI flag, deprecated module, new invariant ("never call X directly"), new directory convention.

Mapping rules — fact type → expected locations

Fact typeShould appear in
Tech stack swap (DB, framework, runtime)CLAUDE.md tech section + relevant rule + setup docs/README
New env var / config keyrunbook + CLAUDE.md (if always-needed) + .env.example
New hard constraint ("never X")a rule (always-loaded), not knowledge
New how-to / command recipeknowledge file + index reference from rule
Deprecated feature/file[DEPRECATED] marker + rule reference removed + replacement noted
Renamed module/functiongrep-replace across all docs + CLAUDE.md if mentioned
New directory or file conventiona rule + CLAUDE.md repository structure section

Procedure

For each new fact:

  1. Determine fact type from the table above
  2. Enumerate expected locations
  3. For each expected location, grep for evidence the fact landed:
    • ✅ Updated: location contains the new fact
    • ❌ Missing: location exists but still describes the old state, or the fact is absent entirely
    • ⚠️ Conflict: location partially updated — old and new state coexist (often worse than fully stale, since it confuses the reader)
  4. Cross-project check: if this repo is depended on by another (monorepo sibling, paired client/server, etc.), flag whether the dependent's docs may need the same update

Output

Append to the Phase 4 report:

## Change Impact Matrix

Source: [user-supplied | git diff | conversation]
Window: <commit range or session start>

### Fact 1: <one-line description, e.g. "DB switched from SQLite to PostgreSQL">

| Expected location | Status | Evidence |
|-------------------|--------|----------|
| CLAUDE.md "Tech Stack" | ❌ Missing | line 42 still reads "SQLite" |
| docs/knowledge/db-schema.md | ⚠️ Conflict | lines 10-30 use Postgres, line 88 still has `sqlite3` import |
| .claude/rules/data-access.md | ✅ Updated | — |
| README.md install steps | ❌ Missing | step 3 still says "create sqlite db" |

Cross-project: agentic_umm-cli depends on this repo's DB schema → flag for review

### Fact 2: ...

Constraints

  • Phase 7 is still read-only — emit the matrix, don't apply edits
  • If no facts are supplied and no recent commits exist, skip silently rather than fabricate
  • Conflicts (⚠️) are higher priority than misses (❌) — call them out first

Constraints

  • Read-only — never modify files. This is an audit tool, not an auto-fixer.
  • Project-agnostic — works with any project that has docs/knowledge/ and .claude/rules/. Does not assume specific file names.
  • Follow the user's language — respond in whichever language the user uses.
  • Deprecated files are not failures — files with [DEPRECATED] markers are expected to exist without rule references. Report them for visibility but don't flag as errors.
  • Threshold, not perfection — the CLAUDE.md compliance check uses a threshold (<=3 direct refs). A summary table mapping rules to knowledge domains is architecture description, not index leakage.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.