Implementing Rate Limiting

1. Configuring Request Rate Limits

TierLimitBurst
Anonymous10 rpm20
Free60 rpm100
Standard600 rpm1000
Premium6000 rpm10000
EnterpriseCustomCustom

Example: Rate limit plugin

plugins:
  - name: rate-limiting
    config:
      second: 10
      minute: 600
      hour: 10000
      policy: redis
      redis_host: redis.internal
      fault_tolerant: true
      hide_client_headers: false

2. Setting Up IP-Based Rate Limiting

KeyRisk
$remote_addrNAT/CGNAT shared IP
X-Forwarded-For firstSpoofable, trust only proxy
Real client IP via PROXYReliable

3. Implementing User-Based Rate Limiting

Example: Limit by JWT sub

plugins:
  - name: rate-limiting
    config:
      limit_by: consumer  # or credential, ip, header
      header_name: X-User-Id
      minute: 100

4. Using API Key Rate Limiting

StrategyUse
Per keyEach key has own quota
Per consumerSum of all keys for user
Per planSubscription tier limits

5. Configuring Burst Limits

SettingDescription
Steady rateSustained reqs/sec
Burst capacityShort spike allowance
Refill rateBucket replenish/sec
Delay vs rejectQueue or 429

6. Setting Up Sliding Window Algorithm

Now: 10:30:45
Window: 60s sliding
Count = (req in [10:29:45, 10:30:45])
Accurate, more memory than fixed window
      

7. Using Fixed Window Algorithm

ProCon
Simple counterBurst at boundary (2x limit possible)
O(1) memoryLess smooth
Easy debug

8. Implementing Token Bucket Algorithm

Example: Token bucket (Java)

class TokenBucket {
  private double tokens;
  private final double capacity, refillPerSec;
  private long lastRefillNanos;

  synchronized boolean tryConsume(int n) {
    long now = System.nanoTime();
    double add = (now - lastRefillNanos) / 1e9 * refillPerSec;
    tokens = Math.min(capacity, tokens + add);
    lastRefillNanos = now;
    if (tokens >= n) { tokens -= n; return true; }
    return false;
  }
}
PropertyValue
BurstYes (up to capacity)
SmoothingRefill-based
Common inAWS, Stripe, Envoy

9. Configuring Rate Limit Headers

HeaderMeaning
X-RateLimit-LimitMax requests in window
X-RateLimit-RemainingRequests left
X-RateLimit-ResetUnix epoch when reset
Retry-AfterSeconds (on 429)
RateLimit-PolicyIETF draft NEW

10. Setting Up Rate Limit Bypass Rules

Bypass ForReason
Internal IPsService-to-service
Health checksOperational paths
Premium tierSLA exception
Admin keysDiagnostic access