Implementing Horizontal Scaling
1. Understanding Scaling Challenges
| Challenge | Reason |
|---|---|
| Stateful sockets | Tied to a node |
| Cross-node fan-out | Pub needs all subs |
| Sticky sessions | Required for some auth models |
| Rolling deploys | Need graceful drain |
2. Using Redis Pub/Sub
Example: Redis fan-out
import { createClient } from "redis";
const pub = createClient(); const sub = pub.duplicate();
await Promise.all([pub.connect(), sub.connect()]);
sub.subscribe("ws-broadcast", (raw) => {
for (const c of wss.clients) if (c.readyState === 1) c.send(raw);
});
function broadcast(msg) { pub.publish("ws-broadcast", JSON.stringify(msg)); }| Tool | Strength |
|---|---|
| Redis Pub/Sub | Simple, low latency |
| Redis Streams | Persistent, replay |
| NATS | High throughput, subjects |
| Kafka | Durable, ordered, replay |
3. Implementing Message Broadcasting
| Pattern | Detail |
|---|---|
| Fan-out by topic | Channel per room |
| User mailbox | Channel per user across devices |
| Server-targeted | Channel per server (rare) |
4. Sharing Session State
| Store | Trade-off |
|---|---|
| Redis | Fast central store |
| JWT | Stateless; hard to revoke |
| DB | Durable; slower |
5. Implementing Sticky Sessions
| LB Method | Detail |
|---|---|
| IP hash | Simple, NAT-prone |
| Cookie | Per-session pinning (L7) |
| Consistent hashing | Stable assignment |
6. Using Message Queue
| Queue | Use Case |
|---|---|
| RabbitMQ | Routing, ACK semantics |
| Kafka | Event streaming, replay |
| SQS | Managed, FIFO option |
| NATS JetStream | Lightweight + durable |
7. Implementing Service Discovery
| Tool | Detail |
|---|---|
| Kubernetes Service / DNS | Built-in |
| Consul / etcd | KV + watch |
| Eureka | JVM ecosystem |
8. Handling Server Addition/Removal
Rolling Deploy Drain
- Mark node unhealthy in LB (stop new conns)
- Broadcast close 1001 to existing clients
- Wait for drain or timeout
- Stop process
- Start new version; readiness probe → traffic
9. Monitoring Cluster Health
| Metric | Use |
|---|---|
| Conn count / node | Balance check |
| CPU / RAM / FD | Saturation |
| Cross-node msg lag | Pub/sub delay |
| Restart count | Stability |
10. Testing Scaling Scenarios
| Test | Goal |
|---|---|
| 10× conns | Vertical limits |
| Node kill | Reconnect storm |
| Rolling deploy | Zero-downtime drain |
| Redis failover | Pub/sub recovery |