Evidence Coverage

An AI will happily answer a three-part question using evidence that covers only one part — and the reader can't tell. The gap is invisible in a fluent paragraph. This skill makes the agent check its own retrieval: does the evidence in hand actually touch every concept the question asked about? If a core concept has no supporting chunk, the honest move is to go back and search, not to improvise.

When to use

After retrieval, before composing the answer. Pairs naturally with query-planning: the concepts the planner produced are exactly what coverage checks against.

The rule

Before answering, map the question's concepts against the retrieved evidence. Score the coverage, list what's missing, and re-retrieve when a core concept has no support.

1. List the concepts the question requires — the same concept groups from planning (drug, outcome, population, comparator), each expanded with its synonyms so a match counts even when the paper uses a different word.
2. Mark each concept covered or missing — covered if at least one retrieved chunk genuinely addresses it (synonym match counts), missing otherwise.
3. Score coverage — the fraction of concepts with support (0 to 1). A high-level number the reader and the agent can act on.
4. Decide: answer or re-retrieve — re-retrieve when coverage is poor or when any core concept (not a peripheral qualifier) is missing. A missing core concept means the answer would be built on a hole.

Doing it well

• Core vs peripheral. A missing comparator may be tolerable; a missing drug or outcome is not. Weight core concepts harder — losing one of them triggers re-retrieval on its own, even if the overall score looks fine.
• Count synonym matches. Coverage that only matches exact words undercounts real support and forces needless re-retrieval. Expand each concept before checking.
• Ignore filler. Generic academic words ("study", "effect", "results") are not concepts and must not count toward coverage.
• State the gap in the answer. If you answer with partial coverage (because re-retrieval found nothing), tell the reader which concept is unsupported. An honest gap beats a confident guess.

Output

Question concepts:  metformin · all-cause mortality · heart failure · type 2 diabetes

  metformin            covered   (3 chunks)
  all-cause mortality  covered   (2 chunks)
  type 2 diabetes      covered   (2 chunks)
  heart failure        MISSING   (0 chunks)   ← core concept

Coverage: 0.75  →  re-retrieve for "metformin AND heart failure AND mortality"

The value is not the score. It is that the agent notices the hole before it writes a confident paragraph that pretends the hole isn't there.