Property-Based Testing

Example tests check the handful of inputs you thought of. Property-based tests check a property against thousands the framework generates — and when one fails, it shrinks the failure to the smallest input that still breaks, handing you a minimal repro instead of a haystack. Reach for it where the input space is too big to enumerate and a clear invariant exists.

Where it earns its keep

• Roundtrips: decode(encode(x)) == x for all x — serializers, parsers, protobuf/JSON codecs, compression.
• Invariants: a sorted list is sorted and a permutation of the input; a balance never goes negative; a state machine never reaches an illegal state.
• Equivalence / oracle: the fast/new implementation agrees with the slow/reference one for all inputs (the parallel-run check in a rewrite).
• Idempotence / commutativity: f(f(x)) == f(x); apply(a,b) == apply(b,a).

Finding the property

The skill is naming the property, not writing the generator. Ask: what must be true regardless of the input? Common shapes — "there and back" (roundtrip), "different path, same result" (oracle/commutativity), "some things never change" (invariant), "doing it twice = doing it once" (idempotence). If you can't state one, the code may not have a crisp contract yet — that's itself a finding.

Shape of a test

from hypothesis import given, strategies as st

@given(st.binary())
def test_roundtrip(payload):
    assert decode(encode(payload)) == payload     # holds for ALL payloads

Stateful histories and read-only observations

For history-bearing stores, generate valid action sequences against a small reference model: repeated updates, ownership changes, observations and reloads. Check required state, proofs and unrelated fields after every action; shrink the sequence, not just its final input.

• Growth: bound encoded bytes, retained-record counts and nesting against the required retained payload at increasing sequence lengths. Include an increasing-payload control. If each event should be retained once, repeated fixed-size changes must not recursively embed earlier histories.
• Observation: when an API promises read-only inspection, assert no store write, replacement or fsync. Unchanged decoded state or bytes alone can hide a same-byte rewrite. Check file identity/mtime or spy persistence calls; account separately for declared cache or telemetry effects.
• Controls: exercise updates alone, transfers alone, interleaved actions, repeated reads and reloads. Retain legitimate history and required evidence; a smaller file obtained by losing records is not a passing result.

If stored state is unexpectedly large, inspect metadata and the serialization and mutation paths first. Reproduce with a tiny owned fixture under event, byte and time limits before loading, copying or hashing the full store. Keep any necessary production inspection within existing ownership and resource bounds; synthetic timings do not measure production latency.

Discipline

• Assert behavior, not the implementation — a property restating the code proves nothing.
• Save the shrunk counterexample as a regression example test — property tests find the bug; a pinned example keeps it dead (pairs with regression-ratchet).
• Constrain generators to the real domain (valid ranges, encodings) so failures are real bugs, not "we never accept that input anyway."
• Don't replace example tests — keep readable examples for the common path; add properties for the space you can't enumerate. They're complementary.