Implementing Rate Limiting Patterns
1. Rate Limiting Pattern
| Goal | Detail |
|---|---|
| Protect | Backend from overload, abuse, runaway clients |
| Limit Dimension | Per-IP, per-user, per-API-key, per-endpoint, global |
| Response When Exceeded | HTTP 429 + Retry-After |
| Headers | X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset |
2. Token Bucket Pattern
| Element | Detail |
|---|---|
| Bucket Capacity | Max burst (e.g., 100 tokens) |
| Refill Rate | Tokens added/sec (e.g., 10/sec) |
| Request Cost | 1 token (or weighted) |
| Behavior | Allows bursts up to capacity |
3. Leaky Bucket Pattern
| Element | Detail |
|---|---|
| Bucket | Queue with fixed size |
| Leak Rate | Fixed processing rate (smooths bursts) |
| Overflow | Reject excess |
| Effect | Constant output rate; no burst tolerance |
4. Sliding Window Pattern
| Variant | Detail |
|---|---|
| Sliding Log | Store timestamp of each request; count last N seconds |
| Sliding Counter | Weighted sum of current + previous fixed window |
| Pros | Fairer than fixed window; no edge bursts |
| Cons | More memory than fixed |
5. Fixed Window Pattern
| Aspect | Detail |
|---|---|
| Mechanism | Counter resets at start of each window (e.g., minute) |
| Pros | Simplest; minimal state |
| Cons | Allows 2× burst around window boundary |
6. Adaptive Rate Limiting Pattern
| Aspect | Detail |
|---|---|
| Mechanism | Limit adjusts based on system health (latency, errors, queue depth) |
| Algorithm | AIMD (additive increase, multiplicative decrease) — TCP-style |
| Library | Netflix concurrency-limits, Envoy adaptive concurrency |
7. Distributed Rate Limiting Pattern
| Approach | Detail |
|---|---|
| Centralized Counter | Redis INCR + EXPIRE; atomic Lua script |
| Token Server | Central authority distributes tokens |
| Local + Sync | Each node has local quota; sync periodically |
| Sticky Routing | Hash key to one node; node owns counter |
Example: Redis Token Bucket (Lua)
local key, capacity, refill, now, cost = KEYS[1], tonumber(ARGV[1]),
tonumber(ARGV[2]), tonumber(ARGV[3]), tonumber(ARGV[4])
local tokens = tonumber(redis.call('hget', key, 't') or capacity)
local last = tonumber(redis.call('hget', key, 'ts') or now)
tokens = math.min(capacity, tokens + (now - last) * refill)
if tokens < cost then return 0 end
redis.call('hmset', key, 't', tokens - cost, 'ts', now)
redis.call('expire', key, 600)
return 18. Client-Based Rate Limiting
| Aspect | Detail |
|---|---|
| Where | Inside client SDK before sending |
| Pros | Avoids wasted RTT to get rejected |
| Cons | Trust required; client may bypass |
| Use With | Server-side as authoritative limit |
9. API Quota Pattern
| Element | Detail |
|---|---|
| Quota Period | Day, month, billing cycle |
| Per Plan | Free / Pro / Enterprise tiers |
| Soft vs Hard | Soft: warn at 80%; Hard: block at 100% |
| Carry-Over | Usually no; resets each period |
10. Rate Limit Response Pattern
| Header / Field | Purpose |
|---|---|
| HTTP 429 | Too Many Requests status |
| Retry-After | Seconds (or HTTP date) until next attempt allowed |
| X-RateLimit-Limit | Max calls per window |
| X-RateLimit-Remaining | Calls left |
| X-RateLimit-Reset | Unix epoch when window resets |
| Body | Problem Details JSON with quota info |