Implementing Load Balancing
1. Using Round Robin Algorithm
| Aspect | Detail |
|---|---|
| Mechanism | Cyclic distribution across instances |
| Pros | Simple, no state |
| Cons | Ignores instance load / latency |
| Use | Homogeneous, stateless services |
2. Using Least Connections Algorithm
| Aspect | Detail |
|---|---|
| Mechanism | Pick instance with fewest active conns |
| Variant | "Least request" (Envoy: P2C — power of two choices) |
| Use | Long-lived requests, varying workloads |
3. Using Weighted Round Robin
| Aspect | Detail |
|---|---|
| Mechanism | Higher-capacity instances get more turns |
| Use | Mixed instance sizes; canary (90/10) |
4. Using IP Hash Algorithm
| Aspect | Detail |
|---|---|
| Mechanism | Hash(client IP) → instance |
| Pros | Sticky w/o cookies |
| Cons | NATs cause skew |
| Variant | Consistent hashing (ring) for cache affinity |
5. Implementing Health-Based Routing
| Mechanism | Detail |
|---|---|
| Active health check | LB probes endpoint |
| Passive (outlier detection) | Eject after N consecutive errors |
| Drain | Stop new traffic before shutdown |
| Slow-start | Ramp traffic to fresh instance |
6. Implementing Sticky Sessions
| Mechanism | Detail |
|---|---|
| Cookie-based | LB issues affinity cookie |
| Application cookie | App sets; LB respects |
| IP affinity | Hash IP |
| Caveat | Hurts elasticity; prefer stateless services + external session store |
7. Using Geographic Load Balancing
| Mechanism | Detail |
|---|---|
| GeoDNS | Route to nearest region by IP |
| Anycast | Same IP advertised globally |
| Latency-based | Pick lowest-RTT POP (Route 53, Cloudflare) |
| Failover | Health-aware regional routing |
8. Implementing Layer 7 Load Balancing
| Capability | Detail |
|---|---|
| HTTP-aware | Path/header routing |
| TLS termination | Decrypt at LB |
| Retry / timeout | Per-route policies |
| Examples | Envoy, Nginx, HAProxy, ALB, Traefik |
9. Using Layer 4 Load Balancing
| Capability | Detail |
|---|---|
| Protocol-agnostic | TCP/UDP forwarding |
| Performance | Higher throughput, lower CPU |
| Examples | NLB, IPVS, HAProxy TCP mode, kube-proxy |
| Trade-off | No HTTP awareness |
10. Handling Load Balancer Failures
| Tactic | Detail |
|---|---|
| Multiple LBs | HA pair w/ VRRP or cloud-managed |
| Health-checked DNS | Route 53 failover records |
| Anycast | Reroute via BGP |
| Client-side LB | Survives proxy failure |
11. Using Load Balancing Tools
| Tool | Best For |
|---|---|
| Envoy | Modern L7, mesh data plane |
| Nginx | L7, mature, scriptable |
| HAProxy | L4/L7, very fast |
| Traefik | K8s/Docker auto-discovery |
| AWS ELB (ALB/NLB) | Managed AWS |
| Cloud Load Balancing (GCP) | Global anycast |
12. Implementing Dynamic Load Balancing
| Signal | Action |
|---|---|
| CPU / RPS / latency | Adjust weights live |
| Endpoint health | Add/remove from pool |
| Locality | Prefer same zone (reduce $/latency) |
| Algorithms | P2C least-request, EWMA |