Working with Distributed Caching
1. Understanding Cache Strategies
Strategy Read Path Write Path Cache-aside (lazy) App reads cache; on miss, load from DB & populate App writes DB; invalidate cache Read-through Cache loads from DB transparently App writes DB Write-through Cache always queried first Cache writes DB synchronously Write-behind Cache always queried Cache writes DB async (risk of loss) Refresh-ahead Predictively reload before TTL —
2. Implementing Read-Through Caching
Example: Caffeine + LoadingCache
LoadingCache< String , User > cache = Caffeine. newBuilder () . maximumSize ( 10_000 ) . expireAfterWrite (Duration. ofMinutes ( 10 )) . recordStats () . build (key -> userRepo. findById (key)); User u = cache. get ( "user:42" ); // loads from DB on miss
3. Implementing Write-Behind Caching
Property Detail Pros Low write latency; write coalescing Cons Data loss on cache crash; ordering issues Use cases High-write metrics, counters Mitigations Persist queue (Redis Streams), shorter flush interval
4. Implementing Cache Invalidation Strategies
Method Detail TTL Auto-expire after N seconds Explicit DELETE App invalidates on write Versioned keys user:42:v3 bumped on updateEvent-driven Subscribe to DB CDC; invalidate Write-through Update cache as part of write
Note: "There are only two hard things in CS: cache invalidation and naming things." — Phil Karlton
5. Handling Cache Coherence
Pattern Detail Single source of truth One cache shard owns key Pub/sub invalidation Broadcast deletes (Redis CLUSTER, ElastiCache) Versioning Check version on read Quorum reads Multi-replica caches
6. Implementing Distributed Cache Partitioning
Method System Consistent hash Memcached client (Ketama), Redis Cluster Hash slots Redis Cluster (16384 slots) Client sharding App hashes key → instance Proxy Twemproxy, Envoy Redis filter
7. Understanding Cache Stampede Problem
Cause Mitigation Many clients miss simultaneously — Solution: Lock / mutex Single loader per key (singleflight) Solution: Probabilistic early refresh XFetch algorithm Solution: Stale-while-revalidate Serve stale; async refresh
8. Implementing Cache Warming
Trigger Detail Startup Preload hot keys from DB Scheduled Refresh top-N every hour Replay logs Replay last hour's reads Predictive ML-based prefetch
9. Implementing Cache Eviction Policies
Policy Behavior Best For LRU Evict least recently used General purpose LFU Evict least frequently used Stable hot set FIFO Evict oldest insertion Streaming workloads Random Random eviction Cheap; near-LRU TinyLFU / W-TinyLFU Frequency sketch + LRU window Caffeine default ARC Adaptive between recency & frequency Mixed workloads
10. Handling Cache Penetration and Breakdown
Problem Cause Solution Penetration Queries for non-existent keys bypass cache Cache negative results; bloom filter pre-check Breakdown Hot key expires; flood to DB Mutex / singleflight; never expire hot keys Avalanche Many keys expire simultaneously Jittered TTLs; staggered refresh