Implementing Rate Limiting Logic
1. Implementing Token Bucket Algorithm
| Property | Detail |
|---|
| Capacity | Max burst |
| Refill rate | Tokens per second |
| Allow if | Tokens ≥ cost; else deny |
| Use | Permits bursts |
Example: Bucket4j
Bucket bucket = Bucket.builder()
.addLimit(Bandwidth.simple(100, Duration.ofMinutes(1)))
.build();
if (!bucket.tryConsume(1)) throw new TooManyRequestsException();
2. Implementing Leaky Bucket Algorithm
| Property | Detail |
|---|
| Queue | FIFO with fixed rate drain |
| Overflow | Reject when full |
| Use | Smooths bursts to constant rate |
3. Implementing Fixed Window Counter
| Property | Detail |
|---|
| Window | e.g., per-minute counter |
| Reset | At window boundary |
| Pro | Simple, low memory |
| Con | Burst at boundary (2x limit possible) |
4. Implementing Sliding Window Log
| Property | Detail |
|---|
| Storage | Sorted set of timestamps per key |
| On request | Remove old entries; check size |
| Pro | Accurate |
| Con | Memory grows with traffic |
5. Implementing Rate Limit Scopes
| Scope | Key |
|---|
| Per IP | Client IP |
| Per user | Authenticated user ID |
| Per API key | API key hash |
| Per tenant | Tenant ID |
| Per endpoint | Method + path pattern |
| Global | Service-wide |
6. Implementing Rate Limit Storage
| Backend | Use |
|---|
| In-memory (Caffeine) | Single instance |
| Redis | Distributed; INCR + EXPIRE |
| Hazelcast | Embedded distributed |
| DynamoDB | Serverless scale |
| Header | Value |
|---|
| X-RateLimit-Limit | Max in window |
| X-RateLimit-Remaining | Remaining in current window |
| X-RateLimit-Reset | Epoch seconds when window resets |
| Retry-After | On 429: seconds to wait |
8. Implementing Rate Limit Exceeded Handler
Example: 429 response
HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1717000060
Content-Type: application/problem+json
{ "type":"https://api.acme.com/errors/rate-limit",
"title":"Too many requests","status":429,
"detail":"Rate limit exceeded; retry after 30s" }
9. Implementing Dynamic Rate Limits
| Source | Detail |
|---|
| Per-tier plans | Free vs Pro vs Enterprise |
| Customer overrides | Per-account custom limits |
| Adaptive | Tighten under high load |
| Feature flag | Toggle limits in real time |
10. Implementing Rate Limit Bypass Logic
| Bypass | For |
|---|
| Internal services | Shared secret / mTLS |
| Health checks | Excluded paths |
| Premium tier | Higher caps |
Warning: Audit bypass usage; misconfigurations are a common DoS vector.
11. Implementing Sliding Window Counter
| Property | Detail |
|---|
| Approach | Two adjacent windows weighted by overlap |
| Pro | Less memory than log; smoother than fixed |
| Trade-off | Approximate |
12. Implementing Distributed Rate Limiting
Example: Redis token bucket (atomic Lua)
// KEYS[1] = bucket key, ARGV = capacity, refillRate, now, requested
String lua = """
local tokens = tonumber(redis.call('hget', KEYS[1], 'tokens') or ARGV[1])
local last = tonumber(redis.call('hget', KEYS[1], 'last') or ARGV[3])
local delta = math.max(0, ARGV[3] - last) * ARGV[2]
tokens = math.min(ARGV[1], tokens + delta)
if tokens < ARGV[4] then return 0 end
tokens = tokens - ARGV[4]
redis.call('hset', KEYS[1], 'tokens', tokens, 'last', ARGV[3])
redis.call('pexpire', KEYS[1], 60000)
return 1
""";