Bayesian Reasoning

Overview

Bayes' theorem: Posterior odds = Prior odds × Likelihood ratio. The strength of belief after evidence equals the strength before, multiplied by how diagnostic the evidence is.

This skill applies Bayesian discipline where people reason about probabilities informally — and failures follow predictable patterns: ignoring the base rate (prior), confusing P(E|H) with P(H|E) (prosecutor's fallacy), over-updating on vivid confirming evidence, treating correlated evidence as independent.

Composes with probabilistic-thinking (Bayes is the operational engine), critical-thinking (formalizes considering alternatives), logical-fallacies (prosecutor's fallacy and base-rate neglect), and first-principles (the prior is bedrock).

When to Use

• High-stakes decision rests on interpreting evidence (medical test, security alert, fraud flag, hiring signal, A/B result)
• "Evidence is consistent with X" is being treated as proof of X
• Base rates ignored — a rare event treated as probable because evidence "looks like" it
• Correlated evidence pieces treated as independent updates
• A benchmark score, AI-capability claim, AI-adoption stat, or AI-capex/valuation figure is being treated as proof without asking how often that signal appears when the underlying claim is false
• Someone says "Bayesian," "prior," "posterior," "base rate," "likelihood ratio," "update"

Not when: genuinely deterministic; no data to anchor a prior; cost of formal update exceeds the value of being more right.

Coaching Novices (Adaptive Front Door)

In Coach mode, respond one step at a time. Each [WAIT] is a hard stop — output that step's question and nothing more.

1. One-line: before believing what the evidence says, ask how common this situation is to start with (prior) and how often evidence would look this way even if the hypothesis is wrong.
2. Check fit. Deterministic problems / no prior anchor → not this lens.
3. Elicit the real claim and the evidence. What exactly are you deciding, and what evidence do you have?

[WAIT — do not advance until user responds]

4. One question at a time: what's the base rate? what would evidence look like if the hypothesis were false? how much should that change your belief?

[WAIT — do not advance until user responds]

5. Close: specific posterior probability, plus what next evidence would change it most.

[WAIT — do not advance until user responds]

The Process

Step 1 — Name hypothesis and alternatives. H vs. not-H must be exhaustive.

Step 2 — Anchor the prior before evidence. P(H) = base rate in the relevant population. Most failures happen here. Examples: disease prevalence (<0.1% for rare conditions); historical fraction of great hires (20-40%); alerts that proved real (<5%); Series A → $1B outcomes (~5%).

Step 3 — Estimate the likelihood ratio. LR = P(E|H) / P(E|not-H). LR > 1 supports H; LR < 1 supports not-H; LR ≈ 1 is non-diagnostic. If you cannot articulate P(E|not-H), you have half the story.

Step 4 — Compute the posterior. Prior odds × LR = Posterior odds → convert back to probability. Example: 0.1% prevalence, LR = 99 → posterior ≈ 9%. A "highly accurate" test on a rare disease still gives 91% chance of no disease on a positive result.

Step 5 — Check evidence dependence. Correlated evidence (three witnesses from the same source) should be treated as ~one piece, not compounded.

Step 6 — Commit and act. State the posterior number, the action threshold, and what next evidence would most move it.

Output: Bayesian Update

# Bayesian Update: <decision>
H / not-H:
Prior P(H):          Source:
Evidence E:
P(E|H):              P(E|not-H):          LR:
Posterior P(H|E):    Interpretation:
Independence check:
Decision threshold:  Action:  Next evidence:

→ Method in Action: Sally Clark Case (1999) · Air France 447 Search (2009–2011) → 2026 lens: Updating Beliefs About AI Capability and Safety From Eval Results (2023–2026)

Pack: Common Bayesian Settings

A "pack" bundles the most common prior and LR patterns for a domain.

Contribute a pack for your domain — see the template at the repo root.

→ Sources: references/sources.md

Common Rationalizations

Note — [D] = designed upfront | [O] = observed in real use. [O] entries are more valuable.

Red Flags

• Prior never named · "Consistent with X" treated as proof of X · Sensitivity without false-positive rate · Correlated evidence summed as independent · Posterior matches vivid story not math · "Highly likely" without a number · Prior chosen after evidence · "Bayesian" invoked without an actual update

Verification

• Hypothesis and alternatives explicitly named and exhaustive
• Prior named with a number and source, before evidence is examined
• P(E|H) AND P(E|not-H) both estimated · Likelihood ratio computed
• Posterior computed numerically · Independence of evidence pieces checked
• Decision threshold and action stated · Next evidence identified

Part of deciqAI Knowledge Skills — 253 open-source thinking skills that make rigor executable for AI agents. The same skills power every deciqAI agent, which runs them autonomously to operate your company. See it run → https://www.deciqai.com/s/bayesian-reasoning · Built by deciqAI · github.com/deciqAI · Contributions welcome.

Agents: latest version & machine-readable metadata → https://www.deciqai.com/s/bayesian-reasoning.json