public class ConsistentHashRing<T> { private final SortedMap<Long, T> ring = new TreeMap<>(); private final int vnodesPerNode; public ConsistentHashRing(int vnodesPerNode) { this.vnodesPerNode = vnodesPerNode; } public void addNode(T node) { for (int i = 0; i < vnodesPerNode; i++) { ring.put(hash(node + "#" + i), node); } } public T getNode(String key) { if (ring.isEmpty()) return null; long h = hash(key); SortedMap<Long, T> tail = ring.tailMap(h); Long target = tail.isEmpty() ? ring.firstKey() : tail.firstKey(); return ring.get(target); } private long hash(String s) { return Hashing.murmur3_128().hashString(s, StandardCharsets.UTF_8).asLong(); }}
Property
Value
Data movement on add/remove
K/N keys (vs K with mod N)
Recommended vnodes
100-256 per physical node
Hash function
MurmurHash3, xxHash, SHA-1
3. Implementing Virtual Nodes (vnodes)
Benefit
Detail
Load balance
Smooths skew across physical nodes
Heterogeneous capacity
Larger nodes get more vnodes
Faster rebalance
Many small transfers, parallel
Used by
Cassandra, DynamoDB, Riak
4. Implementing Hash Slots (Redis Cluster)
Property
Value
Total slots
16384 (2¹⁴)
Slot computation
CRC16(key) mod 16384
Hash tags
{user:42}.profile — only braces hashed for co-location
Resharding
Slot migration via MIGRATE
5. Implementing Range Partitioning
Aspect
Detail
Layout
Sorted key ranges per partition (e.g., [a-f], [g-m])
Strength
Range scans, ordered iteration
Risk
Hot range (sequential keys → one shard)
Auto-split
Splitter divides hot range into two
Used by
HBase, BigTable, Spanner, CockroachDB
6. Handling Partition Rebalancing
Strategy
Mechanism
Pros / Cons
Hash mod N
Reassign all keys
Simple; massive movement
Fixed partitions
10× nodes; reassign whole partitions
Predictable; coarse
Dynamic split
Split when partition exceeds size
Adapts to growth (HBase)
Consistent hash
Move only affected vnodes
Minimal movement
Warning: Avoid automatic rebalancing during peak traffic — schedule or rate-limit transfers.
7. Understanding Hot Spots and Skewed Workloads
Cause
Mitigation
Celebrity user / tenant
Salt key (userId#shardN) and fan-in on read
Sequential timestamps
Hash prefix the time key
Single trending entity
Cache + replicate hot keys
Skewed range
Auto-split hot range
8. Selecting Partition Keys
Criterion
Why
High cardinality
Spreads load
Even access pattern
Avoids hot shards
Aligns with query
Reduces cross-partition scatter-gather
Stable
Avoids re-sharding
9. Handling Cross-Partition Queries
Pattern
Approach
Scatter-gather
Fan out to all partitions, merge
Parallel aggregation
Map-reduce style
Materialized view
Pre-compute by alternate key
Global secondary index
Separate partitioning by index key
10. Implementing Secondary Indexes in Partitioned Systems