agentleFS
Sign inStart free
axiomanticaxiomantic/spellbook/skills/dehallucination/SKILL.md

dehallucination

Skill11 starsChanged 3 months agoMITView on GitHub

Use when verifying that AI-generated claims, references, or assertions are grounded in reality. Triggers: 'does this actually exist', 'is this real', 'did you hallucinate', 'verify these references', 'check if this is fabricated', 'reality check', 'ground truth'. Invoked as quality gate by develop and deep-research. NOT for: verifying technical claims in code (use fact-checking).

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 6 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Verify first: Always check Tier 1-5 sources before accepting a claim.

    line 18
  2. Distinctive:

    Citation required: Every verdict must cite specific evidence.

    line 19
  3. Distinctive:

    Trace spread: When a hallucination is found, check all dependent artifacts.

    line 20
  4. Very specific:

    CoVe on categorization: Run self-interrogation on risk assignments (per skills/shared-references/cove-protocol.md).

    line 59
  5. Distinctive:

    Verify critical first: Check, document, assign confidence, flag HALLUCINATION if contradicted

    line 60
  6. Distinctive:

    When HALLUCINATION detected, all five steps are mandatory.

    line 66
  7. Distinctive:

    If ANY unchecked: complete before returning.

    line 109

≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Dehallucination

<ROLE>Factual Verification Specialist. Adhere to the installed spellbook rule modules.</ROLE>

<analysis>Before verification: artifact under review, context sources, specific concerns, verification scope.</analysis>

<reflection>After verification: claims assessed, confidence levels assigned, hallucinations flagged.</reflection>

Invariant Principles

  1. Verify first: Always check Tier 1-5 sources before accepting a claim.
  2. Citation required: Every verdict must cite specific evidence.
  3. Trace spread: When a hallucination is found, check all dependent artifacts.

Inputs / Outputs

InputRequiredDescription
artifact_pathYesPath to artifact to verify
context_sourcesNoPaths to context files for verification
feedbackNoRoundtable feedback indicating hallucination concerns
OutputTypeDescription
verification_reportInlineClaims and their status
corrected_artifactFileArtifact with hallucinations corrected
confidence_mapInlineMap of claims to confidence levels

Hallucination Categories

CategoryPatternDetection
Fabricated ReferencesCiting non-existent files, functions, APIsCheck if path/function/endpoint exists
Invented CapabilitiesAsserting features that don't existVerify against actual library/framework API
False ConstraintsStating non-existent limitationsCheck if constraint is documented
Phantom DependenciesAssuming unavailable dependenciesCheck requirements, config
Temporal ConfusionMixing planned vs implementedCheck current codebase state

Confidence Levels (Guidelines)

LevelEvidence Required
VERIFIEDDirect evidence (file, code, docs)
HIGHMultiple supporting signals
LOWLimited or conflicting evidence
HALLUCINATIONEvidence contradicts claim

Assessment Process

  1. Extract claims: existence, capability, constraint, relationship statements
  2. Categorize by risk: Critical (security, deps, APIs) > High (implementation) > Medium (config) > Low (docs)
  3. CoVe on categorization: Run self-interrogation on risk assignments (per skills/shared-references/cove-protocol.md). Verify category and risk level accuracy before proceeding.
  4. Verify critical first: Check, document, assign confidence, flag HALLUCINATION if contradicted
  5. Report: Summary stats, critical hallucinations (blocking), warnings, coverage

Recovery Protocol

<CRITICAL> When HALLUCINATION detected, all five steps are mandatory. Skipping propagation check allows false claims to resurface in dependent artifacts. </CRITICAL>
  1. Isolate: Exact text, location, dependents
  2. Trace propagation: Other artifacts referencing this claim
  3. Correct at source: Mark as corrected with reason and evidence
  4. Update dependents: Flag for re-validation
  5. Document lesson: Record in accumulated_knowledge

Example

<example> Artifact claims: "Use the existing UserValidator class in src/validators.py"
  1. Extract claim: existence (UserValidator in src/validators.py)
  2. Check: grep -n "class UserValidator" src/validators.py
  3. Result: class not found
  4. Assessment: CLAIM: "UserValidator exists" | TYPE: existence | EVIDENCE: grep found no match | CONFIDENCE: HALLUCINATION
  5. Recovery: Correct to "Create new UserValidator class" or find actual validator location</example>

Integration with Develop Workflow

Invoke after: gathering-requirements (verify codebase claims), design-exploration (verify technical capabilities), writing-plans (verify implementation assumptions), roundtable flags hallucination concerns.

<FORBIDDEN> - Accepting claims without checking evidence - Assigning VERIFIED without verification - Silently correcting hallucinations (must document) - Proceeding with unresolved HALLUCINATION findings - Skipping propagation check for detected hallucinations </FORBIDDEN>

Self-Check

  • Critical claims extracted and categorized
  • Verification attempted for critical/high-risk claims
  • Confidence levels assigned with evidence
  • HALLUCINATION findings have corrections
  • Propagation checked
  • Report generated
<CRITICAL> If ANY unchecked: complete before returning. Do not return a partial verification report. </CRITICAL>

<FINAL_EMPHASIS> Hallucinations are confident lies. Every claim needs evidence or explicit uncertainty. When you find one, trace its spread and correct at source. The development workflow depends on factual grounding. </FINAL_EMPHASIS>

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free