Multi-Layer Caching Strategy

A cache is a bet that staleness is cheaper than recomputation. The two hard problems are invalidation and the failure mode when the bet is wrong; the speedup is the easy part.

Define the staleness contract before adding the cache

1. Tolerable staleness per data type: must-be-fresh (auth, balances) vs seconds-stale (feeds) vs minutes-stale (config). This dictates everything downstream.
2. Read/write ratio and key cardinality: caching only pays when reads dominate and the working set fits the tier.
3. Coherence requirement: can two users see different values simultaneously? If not, you need single-source invalidation, not per-node TTLs.

Write policy — pick deliberately

• Cache-aside (lazy): app reads cache, on miss loads DB and populates. Simple; first request slow; risk of stale on concurrent write.
• Write-through: write hits cache + DB synchronously. Fresh reads, slower writes.
• Write-behind: write cache, flush to DB async. Fast writes, risk of data loss and ordering bugs on crash.
• Read-through: cache owns loading. Centralizes logic; couples cache to data source.

Invalidation — where correctness dies

• TTL is eventual coherence; bounded staleness but a thundering-herd on expiry.
• Explicit invalidation is precise but races with concurrent repopulation (the classic "delete then someone reads stale and repopulates" bug — invalidate after the write commits, and consider versioned keys).
• Versioned/immutable keys sidestep invalidation: content-addressed keys never go stale; you change the key, not the value.

Failure modes (design for these explicitly)

• Stampede / cache stampede: a hot key expires and N requests all recompute. Mitigate with request coalescing (single-flight), probabilistic early expiry, or locks.
• Cold start / cache flush: after deploy or eviction, the origin gets full load it can't handle. The cache became a load-bearing wall — size the origin for cache-miss traffic or warm it.
• Negative caching gap: not caching "not found" lets a hot missing key hammer the DB.
• Inconsistent layers: CDN serves what app already invalidated. Coherence must span layers or be explicitly bounded per layer.

Adversarial validation

• Kill the cache tier entirely: does the origin survive the resulting load, or does the cache outage become a total outage?
• Concurrent write + read on the same key: what's the worst stale value a user can observe, and for how long?
• A single key gets 100x traffic (celebrity problem): does one hot key melt one shard?

Non-obvious risks

• Caching converts a latency problem into a consistency problem plus a capacity cliff.
• Per-node in-process caches multiply staleness windows and are invisible to central invalidation.
• High hit-rate hides origin fragility until the day it doesn't.

Output structure

Deliver: per-data-type staleness contract, the layer map with write/invalidation policy per layer, the stampede + cold-start protection, the origin's survivability under full miss load, and the worst-case observed staleness.