Implementing Scalability Patterns
1. Horizontal Scaling Pattern
| Aspect | Detail |
|---|---|
| Mechanism | Add more instances; distribute load via LB |
| Requires | Stateless services or externalized state |
| Pros | Near-linear scaling; commodity hardware |
| Cons | Coordination overhead; stateful workloads harder |
2. Vertical Scaling Pattern
| Aspect | Detail |
|---|---|
| Mechanism | Bigger instance (more CPU/RAM) |
| Pros | No code change; simple |
| Cons | Hardware ceiling; downtime to resize; cost spikes |
| Use For | Stateful services (DBs); legacy apps |
3. Load Balancing Pattern
| Algorithm | Use |
|---|---|
| Round Robin | Uniform distribution; simple |
| Least Connections | Long-lived connections |
| Least Response Time | Heterogeneous latencies |
| Weighted | Mixed instance sizes |
| Consistent Hash | Cache affinity, session stickiness |
| Power of Two Choices | Sample 2; pick less loaded — near-optimal |
4. Auto-Scaling Pattern
| Trigger | Detail |
|---|---|
| CPU Utilization | Most common; lagging indicator |
| Request Rate | Direct demand signal |
| Queue Depth | For async workers (KEDA) |
| Custom Metrics | p99 latency, errors |
| Scheduled | Time-of-day patterns |
| Predictive | ML forecast |
Example: K8s HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: order-service }
spec:
scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: order-service }
minReplicas: 3
maxReplicas: 50
metrics:
- type: Resource
resource: { name: cpu, target: { type: Utilization, averageUtilization: 70 } }5. Stateless Service Pattern
| Rule | Detail |
|---|---|
| No In-Memory Session | Externalize to Redis / DB / cookie |
| No Local Files | Use object store |
| Idempotent | Any instance can serve any request |
| Benefit | Trivial horizontal scaling, fast restart |
6. Database Sharding Pattern
| Strategy | Detail |
|---|---|
| Hash-Based | hash(key) → shard |
| Range-Based | Key ranges per shard |
| Directory-Based | Lookup table for shard mapping |
| Geo-Based | Shard by region |
| Tools | Vitess, Citus, MongoDB sharding, Cassandra |
7. Read Replica Pattern
| Aspect | Detail |
|---|---|
| Topology | 1 primary (writes) + N replicas (reads) |
| Replication | Async (default) or sync |
| Read Routing | Smart client / proxy splits reads |
| Trade-off | Replica lag → eventual consistency |
8. Queue-Based Load Leveling Pattern
| Aspect | Detail |
|---|---|
| Definition | Use queue between bursty producer and steady consumer |
| Effect | Smooths traffic; consumer scales independently |
| Auto-Scale | Scale consumers based on queue depth (KEDA) |
9. Partitioning Pattern
| Type | Use |
|---|---|
| Functional (Vertical) | Different services own different data domains |
| Horizontal (Sharding) | Same schema split by key |
| Topic Partitioning (Kafka) | Parallel consumers per partition |
| Hash + Range Hybrid | Cassandra primary key |
10. Elastic Scaling Pattern
| Aspect | Detail |
|---|---|
| Definition | Scale up AND down automatically with demand |
| Cost Benefit | Pay for what you use |
| Scale-Down Care | Drain in-flight requests; respect graceful shutdown |
| Cooldown | Prevent flapping |