Building Scalability Patterns
1. Understanding Horizontal vs Vertical Scaling
| Type | How | Limit |
|---|---|---|
| Vertical | Bigger machine | Hardware ceiling, cost^2 |
| Horizontal | More machines | Coordination cost; needs stateless or partitioned design |
| Diagonal | Vertical until cost-prohibitive, then horizontal | Common pragmatic path |
2. Implementing Stateless Service Design
| Principle | Detail |
|---|---|
| No per-request server state | Externalize to DB / cache / token |
| Idempotent endpoints | Safe retries |
| Configuration externalized | Env vars, config service |
| 12-Factor App | Industry baseline |
3. Implementing Database Sharding
| Strategy | Detail |
|---|---|
| Horizontal (row) | Split rows across shards by key |
| Vertical (column) | Split columns by access pattern |
| Functional | Different tables to different DBs |
| Geo / tenant | By region or tenant id |
| Tools | Vitess, Citus, MongoDB sharded cluster |
4. Implementing Read-Write Splitting
| Pattern | Detail |
|---|---|
| Writes → leader | Single source of truth |
| Reads → replicas | Scale horizontally |
| Replication lag handling | Read-your-writes via leader read after write |
| Tools | ProxySQL, PgBouncer, RDS proxy |
5. Implementing CQRS for Scalability
| Benefit | Detail |
|---|---|
| Independent scaling | Reads and writes scale separately |
| Optimized models | Read views denormalized for queries |
| Multi-store | Write to RDBMS, read from ES / Redis |
| Cost | Eventual consistency between sides |
6. Implementing Asynchronous Processing
| Pattern | Detail |
|---|---|
| Queue + worker | Decouple ingestion from processing |
| Event-driven | React to events; loose coupling |
| Async APIs | 202 Accepted + status endpoint |
| Use cases | Email, image processing, ML inference |
7. Implementing Batch Processing
| Aspect | Detail |
|---|---|
| Frameworks | Spark, Flink batch, Beam, MapReduce |
| Scheduler | Airflow, Argo Workflows, Prefect, Dagster |
| Window | Hourly/daily/weekly tumbling |
| Ideal for | ETL, reports, training pipelines |
8. Implementing Stream Processing for Scale
| Element | Detail |
|---|---|
| Partitioned input | Kafka topic per source |
| Parallel operators | Per-partition tasks |
| Stateful with checkpoints | Recover from failure |
| Tools | Flink, Kafka Streams, Spark Structured |
9. Understanding Auto-Scaling Strategies
| Type | Detail |
|---|---|
| Reactive (CPU/RPS) | HPA based on metric thresholds |
| Predictive | Forecast-based pre-scale (AWS Predictive) |
| Scheduled | Time-based (e.g., business hours) |
| Custom metrics | Queue depth, lag (KEDA) |
| Vertical (VPA) | Right-size pod resources |
| Cluster autoscaler | Add/remove nodes |
10. Implementing Resource Pooling
| Pool | Tuning |
|---|---|
| DB connections | HikariCP; size = 2 × cores typical |
| HTTP clients | Per-route max; keep-alive |
| Threads | Bounded; avoid unbounded executor |
| Object pooling | Apache Commons Pool |
11. Implementing CDN for Static Content
| Feature | Detail |
|---|---|
| Edge caching | POPs near users |
| Cache-Control headers | public, max-age=31536000, immutable |
| Cache key | URL + selected query/headers |
| Invalidation | Purge by URL or tag |
| Tools | CloudFront, Cloudflare, Fastly, Akamai |