agentleFS
TheArcitectTheArcitect/claude-skills/coherency-audit/SKILL.md

coherency-audit

Skill2 starsChanged 9 months agoMITView on GitHub

Detect and surface contradictions between Claude's synthesized profile and actual conversation history. Prevents stale data from compounding across conversations. Trigger on "run coherency audit", "check coherency", "audit your memory", or before making claims about people, employers, case statuses, or project statuses where wrong facts have real consequences.

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 9 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When layers conflict, Claude defaults to the synthesized profile because it appears first in context.

    line 19
  2. Distinctive:

    Start of conversations involving professional communications, legal strategy, or job applications where wrong facts have consequences

    line 25
  3. Distinctive:

    Before making claims about people's employers, roles, or affiliations

    line 28
  4. Distinctive:

    When conversation_search results contradict profile data

    line 29
  5. Distinctive:

    When the user references something Claude doesn't see in the profile

    line 30
  6. Distinctive:

    Read the synthesized profile for each category.

    line 53
  7. Distinctive:

    If unsure whether data is stale, ASK rather than assuming the profile is correct

    line 72
  8. Distinctive:

    Memory edits override the profile: If there's a conflict, edits win — they're more recent and user-directed.

    line 80
  9. Distinctive:

    Update this skill whenever major life events occur — case resolutions, job changes, project completions.

    line 85

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Coherency Audit Skill

Purpose

Detect and surface contradictions between Claude's synthesized profile (userMemories) and actual conversation history. Prevents stale data from compounding silently across conversations.

The Problem

Claude's memory system has three layers:

  1. Synthesized profile (userMemories) — generated periodically, goes stale
  2. Memory edits (user_edits) — more current but incomplete
  3. Conversation history — the actual ground truth

When layers conflict, Claude defaults to the synthesized profile because it appears first in context. Stale or incorrect claims propagate silently and compound. The most dangerous errors sound plausible.

When to Trigger

Mandatory

  • User says "run coherency audit" or "check coherency" or "audit your memory"
  • Start of conversations involving professional communications, legal strategy, or job applications where wrong facts have consequences

Recommended

  • Before making claims about people's employers, roles, or affiliations
  • When conversation_search results contradict profile data
  • When the user references something Claude doesn't see in the profile
  • At least once per week during active daily use

Execution

Step 1: Identify Auditable Categories

Categories most likely to go stale:

  1. Employment — employers, roles, titles (yours and people you discuss)
  2. Case/project status — open, closed, won, lost, abandoned, active
  3. Current projects — what's actually active vs. what profile says
  4. Location — where relevant
  5. Relationship details — names, roles, affiliations

Step 2: Search for Evidence

For each category, run targeted searches:

conversation_search("[person] employer job work [company]")
conversation_search("[case] ruling outcome status")
conversation_search("current project building working on")
recent_chats(n=5)

Step 3: Compare to Profile

Read the synthesized profile for each category. Compare to conversation evidence. Flag every discrepancy.

Step 4: Output the Audit

COHERENCY AUDIT — [date]

VERIFIED: [category] — profile matches evidence
STALE: [category]
   Profile says: [X]
   Evidence shows: [Y]
   Source: [chat date or reference]
   Recommend: [suggested memory edit]

MISSING: [category] — profile has no entry, evidence shows [Z]
   Recommend: [suggested memory edit]

Step 5: Fix

  • Apply memory edits to correct stale entries immediately
  • If unsure whether data is stale, ASK rather than assuming the profile is correct
  • Memory edits override the synthesized profile

Critical Rules

  1. Never trust the synthesized profile over conversation evidence. The profile is a periodic summary. Conversations are ground truth.
  2. When in doubt, search. A 30-second search is cheaper than a wrong claim that compounds across threads.
  3. The most dangerous errors sound plausible. That's why they persist.
  4. Memory edits override the profile. If there's a conflict, edits win — they're more recent and user-directed.
  5. Stale status is as dangerous as stale facts. Describing a won case as "ongoing litigation" changes how Claude frames every response.

Maintenance

Update this skill whenever major life events occur — case resolutions, job changes, project completions. The user may direct updates. Claude should also proactively suggest updates when evidence accumulates.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.