Uncertainty and Explainability Patterns

The goal of an explanation is not to look transparent. It is to help the user decide whether to rely on the output.

What this skill is

A pattern-selection workflow that matches how uncertainty and reasoning are shown to who is looking and what is at stake. It profiles the audience and decision, picks from a catalog of confidence, provenance, and reasoning patterns, sets display thresholds, and checks the design against common anti-patterns. The output is a specification a designer and engineer can implement consistently across a product.

What it solves

• Confidence percentages that look precise but are not calibrated to real accuracy
• Citation lists that link to sources which do not support the claim
• Long reasoning dumps that users skip, adding noise without improving decisions
• The same explanation shown to a novice consumer and an expert analyst
• Users over-relying on AI output in high-stakes moments, or ignoring it in low-stakes ones

When to invoke

• Deciding whether and how to display a confidence score or probability
• Designing citations and source display for a search or retrieval answer
• Explaining a model recommendation, ranking, classification, or flag
• Regulatory or internal policy requires explainability for automated decisions
• Research shows users accept wrong answers, or redo work the AI did correctly

Phase 1: Profile the audience and decision

The core question per decision: what does the user need to know to decide whether to rely on this output?

Phase 2: Measure before you show

Uncertainty display is only honest if the underlying signal is real.

Calibration check: for outputs labeled "High confidence," measured accuracy should be within 5 points of the implied rate
If uncalibrated, collapse to 2-3 categorical bands or remove the confidence display

Phase 3: Select confidence patterns

Threshold rule: define the confidence level below which the feature abstains or asks rather than answers. For high-stakes decisions this matters more than any display choice.

Phase 4: Select provenance and reasoning patterns

Match by profile:

Layer it: one glanceable signal by default, evidence one click away, full trace for those who want it.

Phase 5: Run the anti-pattern check

Output

• Audience and decision profile per surface
• Signal inventory with calibration status
• Selected confidence, provenance, and reasoning patterns per surface
• Abstain-or-ask thresholds for high-stakes decisions
• Layering spec (default, one click, full trace)
• Anti-pattern checklist with pass or fail and fixes

Operating rules

Always

• Verify calibration before showing any numeric confidence
• Tie every citation to the specific claim it supports
• Set a threshold below which the system abstains or asks
• Layer explanations: glanceable first, detail on demand
• Test whether the pattern changes user reliance in the right direction

Never

• Display model self-reported certainty as if it were a measured probability
• Use more decimal places than the evidence supports
• Count the number of citations as a proxy for answer quality
• Show the same explanation depth to every audience
• Rely on a generic disclaimer as the trust mechanism