Isolate resources (threads, connections) per dependency/tenant
Benefit
One slow dependency can't exhaust all threads
Tools
Resilience4j Bulkhead, separate connection pools
2. Bulkhead Pool Configuration
Param
Typical
maxConcurrentCalls
10-50 per dependency
maxWaitDuration
0 (fail fast) or short (100ms)
queueCapacity
0 or small for thread pool bulkhead
type
Semaphore (lighter) or ThreadPool (better isolation)
3. Retry Pattern
Aspect
Detail
When
Transient failures only (5xx, timeouts, 429)
Never Retry
4xx (validation, auth), non-idempotent without idempotency key
Backoff
Exponential with jitter
Max Attempts
3-5 typical
Example: Exponential Backoff + Jitter
RetryConfig cfg = RetryConfig.custom() .maxAttempts(4) .intervalFunction(IntervalFunction.ofExponentialRandomBackoff( Duration.ofMillis(200), 2.0, 0.5)) .retryOnException(e -> e instanceof IOException || e instanceof TimeoutException) .build();
Warning: Naive retries cause retry storms during outages. Always use jitter and cap retries — and disable retries for upstream errors that came from a downstream retry.