Bias Analysis

When to use

• User compares options (products, tools, vendors, engines, libraries) from field evidence
• A study cites forums, reviews, surveys, breakdown statistics, or benchmarks
• User asks "is X more reliable than Y" or "why does everyone say X is better"
• A comparative document needs a bias section, or an existing one looks ad hoc

Overview

Field evidence carries systematic distortions. Name them, test them, and correct the conclusion; never just list them. Every verdict on a bias needs evidence, like any other claim: the evidence-based-analysis skill owns citation rules.

Reference: biases.md. Sweep the full checklist; never pick biases from memory.

Workflow

1. Inventory the sources. Tag each one: fleet statistic, survey, forum anecdote, vendor claim, professional test. The tag sets its weight.
2. Sweep the checklist in biases.md against every option, not only the favourite. A bias analysis that only weakens one side is itself biased.
3. For each candidate bias, record four fields: mechanism, direction (which option it inflates), evidence for or against, magnitude (weak, moderate, strong).
4. Give a verdict per bias: confirmed, plausible, or rejected. Confirmed requires cited evidence, not a plausible mechanism. A mechanism alone caps at plausible.
5. Separate genuine causal advantages from perception effects. List them per option under "real advantages, not biases". This prevents over-correction.
6. Re-state the comparison after correction. Say which claims survive, which flip, and which stay undecidable. A closing line must follow from the table, never from hand-waving.

Verdict table

One row per bias, all four fields filled:

Not: "Volume bias: X has a bigger fleet, so more reported failures. Confirmed." Yes: "Volume bias: plausible — the mechanism holds but no fleet-size figures found; failure counts stay uncorrected."

Symmetry check

Before closing, ask once per option: which biases inflate this one? Common misses: cheaper repairs get fixed instead of reported; the study language over-represents a local brand; enthusiast forums self-select complainers on every side; the newer option has less time to fail.

Boundaries

This skill weighs external evidence: reviews, statistics, reports. Claims about code stay with evidence-based-analysis. It never rejects evidence because a bias exists; it grades how much the bias moves the conclusion.