Implementing Rate Limiting
1. Using Token Bucket Algorithm
| Param | Meaning |
|---|---|
| Rate (r/s) | Sustained throughput |
| Burst | Max bucket size; absorbs short spikes |
| Take | Consume token before allowing call |
2. Implementing Rate Limit Interceptor
Example: Global rate limit
import "golang.org/x/time/rate"
limiter := rate.NewLimiter(rate.Limit(1000), 200) // 1000 r/s, burst 200
func RateLimit(ctx context.Context, req any, info *grpc.UnaryServerInfo,
h grpc.UnaryHandler) (any, error) {
if !limiter.Allow() {
return nil, status.Error(codes.ResourceExhausted, "rate limit")
}
return h(ctx, req)
}3. Using golang.org/x/time/rate Package
| API | Detail |
|---|---|
rate.NewLimiter(r, b) | Create limiter |
Allow() | Non-blocking |
Wait(ctx) | Block until token (or ctx done) |
Reserve() | Returns delay until next allow |
4. Setting Rate Limit per Client
Example: Per-API-key limiter
var limiters sync.Map // key → *rate.Limiter
func getLimiter(k string) *rate.Limiter {
l, ok := limiters.Load(k)
if !ok {
l, _ = limiters.LoadOrStore(k, rate.NewLimiter(50, 10))
}
return l.(*rate.Limiter)
}5. Returning ResourceExhausted Status
| Detail | Value |
|---|---|
| Status code | codes.ResourceExhausted |
| Hint header | Include retry-after trailer |
| Error detail | errdetails.RetryInfo |
6. Implementing Sliding Window
| Variant | Detail |
|---|---|
| Fixed window | Counter resets each interval |
| Sliding window log | Store timestamps; precise but memory-heavy |
| Sliding window counter | Weighted average of two windows — common balance |
7. Using Redis for Distributed Limiting
Example: Redis token bucket (Lua)
EVAL "local n = redis.call('INCR', KEYS[1]) \
if n == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]) end \
return n" 1 user:u123:rl 60| Library | Detail |
|---|---|
| go-redis/redis_rate | Built-in algos |
| Envoy global RL | Sidecar pattern |
8. Adding Rate Limit Headers
| Header | Purpose |
|---|---|
x-ratelimit-limit | Max in window |
x-ratelimit-remaining | Remaining tokens |
x-ratelimit-reset | Seconds until reset |
9. Implementing Per-Method Limits
| Approach | Detail |
|---|---|
| Method → limiter map | Different rate per RPC |
| Cost weighting | Expensive methods consume more tokens |
10. Configuring Burst Size
| Guideline | Detail |
|---|---|
| Burst ≈ 10–25% of rate | Absorbs short spikes |
| High burst | Tolerates bursty workloads but risks overload |