enumerate-first

The rule

Generate and measure instead of arguing. Report counts, not adjectives.

Reachability arguments are cheap to make and almost always wrong, because they reason from the design you intended rather than the code you have.

Why this exists

A tie that a code review called narrow and probably unreachable produced 1207 violations in three defect classes when 35,888 generated cases were fuzzed: none was the hypothesised one, and most sat in a component the argument had dismissed. A 625-configuration grid search once tuned a selector that executed zero times. Each was one measurement away.

Procedure

1. Define the space. What varies — inputs, orderings, configurations, geometries? Write it down; the gaps in your list are where defects hide.
2. Generate broadly, seeded. Random with a fixed seed, or exhaustive if the space is small. Reproducibility matters more than elegance.
3. Run the property, count the failures. Print the count and a handful of concrete examples.
4. Classify before fixing. Group violations — three defect classes look like one bug until you group them.
5. Keep it as a gate if it is fast enough, and mutation-test it (see prove-invariant). If the gate passes on the mutant, tighten it and return to Step 3.

Cheap before expensive

Order probes by cost and let the cheap one filter for the expensive one:

• Run the check that needs no oracle, no JVM, no network first.
• Escalate only the survivors to the expensive check.
• Batch the expensive check once over all survivors rather than per item.

Before tuning anything

Confirm the decision point executes: count its calls for the target inputs (spy-site). Documented central logic called zero times is common, not exotic.

Red flags