Implementing Rate Limiting
1. Configuring Request Rate Limits
| Tier | Limit | Burst |
|---|---|---|
| Anonymous | 10 rpm | 20 |
| Free | 60 rpm | 100 |
| Standard | 600 rpm | 1000 |
| Premium | 6000 rpm | 10000 |
| Enterprise | Custom | Custom |
Example: Rate limit plugin
plugins:
- name: rate-limiting
config:
second: 10
minute: 600
hour: 10000
policy: redis
redis_host: redis.internal
fault_tolerant: true
hide_client_headers: false
2. Setting Up IP-Based Rate Limiting
| Key | Risk |
|---|---|
$remote_addr | NAT/CGNAT shared IP |
X-Forwarded-For first | Spoofable, trust only proxy |
| Real client IP via PROXY | Reliable |
3. Implementing User-Based Rate Limiting
Example: Limit by JWT sub
plugins:
- name: rate-limiting
config:
limit_by: consumer # or credential, ip, header
header_name: X-User-Id
minute: 100
4. Using API Key Rate Limiting
| Strategy | Use |
|---|---|
| Per key | Each key has own quota |
| Per consumer | Sum of all keys for user |
| Per plan | Subscription tier limits |
5. Configuring Burst Limits
| Setting | Description |
|---|---|
| Steady rate | Sustained reqs/sec |
| Burst capacity | Short spike allowance |
| Refill rate | Bucket replenish/sec |
| Delay vs reject | Queue or 429 |
6. Setting Up Sliding Window Algorithm
Now: 10:30:45
Window: 60s sliding
Count = (req in [10:29:45, 10:30:45])
Accurate, more memory than fixed window
7. Using Fixed Window Algorithm
| Pro | Con |
|---|---|
| Simple counter | Burst at boundary (2x limit possible) |
| O(1) memory | Less smooth |
| Easy debug | — |
8. Implementing Token Bucket Algorithm
Example: Token bucket (Java)
class TokenBucket {
private double tokens;
private final double capacity, refillPerSec;
private long lastRefillNanos;
synchronized boolean tryConsume(int n) {
long now = System.nanoTime();
double add = (now - lastRefillNanos) / 1e9 * refillPerSec;
tokens = Math.min(capacity, tokens + add);
lastRefillNanos = now;
if (tokens >= n) { tokens -= n; return true; }
return false;
}
}
| Property | Value |
|---|---|
| Burst | Yes (up to capacity) |
| Smoothing | Refill-based |
| Common in | AWS, Stripe, Envoy |
9. Configuring Rate Limit Headers
| Header | Meaning |
|---|---|
X-RateLimit-Limit | Max requests in window |
X-RateLimit-Remaining | Requests left |
X-RateLimit-Reset | Unix epoch when reset |
Retry-After | Seconds (on 429) |
RateLimit-Policy | IETF draft NEW |
10. Setting Up Rate Limit Bypass Rules
| Bypass For | Reason |
|---|---|
| Internal IPs | Service-to-service |
| Health checks | Operational paths |
| Premium tier | SLA exception |
| Admin keys | Diagnostic access |