Implementing Load Balancing
1. Configuring Round Robin Algorithm
Property Value Distribution Even, rotational State Stateless counter Best for Homogeneous backends Limitation Ignores load/latency
Example: Round-robin upstream
upstream backend { server svc-1.internal:8080; server svc-2.internal:8080; server svc-3.internal:8080; }
2. Using Least Connections Algorithm
Property Value Distribution Pick lowest active conns State Per-target counter Best for Long-lived requests NGINX directive least_conn;
3. Using IP Hash Algorithm
Example: IP hash for session affinity
upstream backend { ip_hash ; server svc-1.internal:8080; server svc-2.internal:8080; }
Property Value Hash key Client IP Stickiness Same IP → same backend Gotcha NAT, mobile users skew
4. Implementing Weighted Load Balancing
Target Weight Share large.svc 4 50% medium.svc 2 25% small.svc 2 25%
5. Configuring Consistent Hashing
Property Value Algorithm Ring hash (Ketama/Maglev) Hash input Header, cookie, query, path Re-balance cost O(1/N) keys move on change Best for Cache locality, sharding
Example: Consistent hash by user ID (Envoy)
load_balancing_policy : policies : - typed_extension_config : name : envoy.load_balancing_policies.ring_hash typed_config : "@type" : type.googleapis.com/envoy.extensions.load_balancing_policies.ring_hash.v3.RingHash minimum_ring_size : 1024 hash_policy : - header : { header_name : x-user-id }
6. Implementing Sticky Sessions
Method Mechanism Cookie insert GW adds SERVERID cookie Cookie prefix Modify existing app cookie App cookie Hash app-set cookie Header Hash custom header
Warning: Sticky sessions break horizontal scaling and graceful drain. Prefer stateless sessions (JWT, Redis).
7. Setting Load Balancing Weights
Scenario Weight Setup Heterogeneous instances 2x CPU → 2x weight Canary Stable=99, canary=1 Drain Target weight → 0 Cost balancing Spot=70, on-demand=30
8. Setting Up Active Health Checks
Example: Envoy active health check
health_checks : - timeout : 1s interval : 5s unhealthy_threshold : 3 healthy_threshold : 2 http_health_check : path : /healthz expected_statuses : [{ start : 200 , end : 299 }]
9. Configuring Passive Health Checks
Trigger Action 5 consecutive 5xx Eject 30s Connection reset Eject 10s Timeout Eject 30s Local origin failure Eject + circuit break
10. Configuring Failover Behavior
Primary OK → Route to primary
Primary FAIL → Retry on primary (n=2)
Still FAIL → Switch to secondary pool
Secondary OK → Stay until primary healthy 30s
Strategy Use Active-active Both serve traffic Active-passive Standby only on failure Priority groups Tier 1 → tier 2 fallback