Managing Timeout and Deadline Patterns
1. Request Timeout Pattern
| Aspect | Detail |
|---|---|
| Definition | Cap on time to receive response |
| Default | NEVER infinite |
| Set Where | HTTP client config; per-call override |
| Tuning | p99 latency × 2-3, capped by user-tolerable wait |
2. Connection Timeout Pattern
| Aspect | Detail |
|---|---|
| Definition | Cap on TCP/TLS handshake |
| Typical | 1-3 seconds |
| Symptom If Missing | Connections hang on bad DNS/firewall |
3. Deadline Propagation Pattern
| Aspect | Detail |
|---|---|
| Definition | Pass remaining time budget to downstream |
| Header | gRPC grpc-timeout, custom X-Deadline-Ms |
| Effect | Downstream rejects if can't complete in time |
| Benefit | Avoids work for already-timed-out caller |
Example: gRPC Deadline (Go)
ctx, cancel := context.WithTimeout(parentCtx, 500*time.Millisecond)
defer cancel()
resp, err := client.GetOrder(ctx, &pb.OrderRequest{Id: id})
// Context.deadline propagates to downstream calls automatically4. Cascading Timeout Pattern
| Rule | Detail |
|---|---|
| Inner ≤ Outer | Each downstream timeout shorter than parent |
| Buffer | Leave 50-100ms for serialization, network |
| Avoid | Inner = Outer (parent times out before retry possible) |
5. Adaptive Timeout Pattern
| Aspect | Detail |
|---|---|
| Mechanism | Timeout based on rolling p99 of latency |
| Benefit | Auto-adjusts to system state |
| Risk | Slow drift hides real degradation |
6. Long-Running Operation Pattern
| Approach | Detail |
|---|---|
| 202 Accepted + Polling | Return job ID; client polls /jobs/{id} |
| Webhook | Client provides callback URL |
| WebSocket / SSE | Push progress updates |
| Async Result | Result in queue / event |
Example: Long-Running Operation Response
HTTP/1.1 202 Accepted
Location: /jobs/abc123
Retry-After: 5
{ "jobId": "abc123", "status": "PROCESSING", "checkUrl": "/jobs/abc123" }7. Timeout Recovery Pattern
| Step | Action |
|---|---|
| Cancel | Propagate cancellation to free downstream resources |
| Cleanup | Release connections, threads, locks |
| Compensate | If side-effects committed, run reversal |
| Log | Timeout with context for debugging |
8. Timeout Budget Pattern
| Concept | Detail |
|---|---|
| Budget | Total time available at edge (e.g., 2s) |
| Spend | Each hop subtracts elapsed |
| Remaining | Passed in deadline header |
| Reject Early | If remaining < estimated work, fail fast |
9. Asynchronous Timeout Pattern
| Aspect | Detail |
|---|---|
| Use | Non-blocking timeout in reactive/async code |
| Java | CompletableFuture.orTimeout() (Java 9+) |
| Reactor | Mono.timeout(Duration) |
| Cancellation | Triggers downstream cleanup |
10. Deadline Context Pattern
| Aspect | Detail |
|---|---|
| Pattern | Carry deadline in request context (gRPC ctx, OTel baggage) |
| Auto-Propagation | Library injects/extracts on every call |
| Benefit | Whole call tree shares one deadline |