Managing Inter-Service Communication
1. Using Synchronous Communication
| Protocol | Best For | Trade-off |
|---|---|---|
| REST/HTTP | Public APIs, broad clients | Verbose, text-based |
| gRPC | Internal high-perf, streaming | Browser support needs gRPC-Web |
| GraphQL | Aggregating UIs | Caching complexity |
| Drawback | Tight coupling; caller waits; failures cascade | |
2. Using Asynchronous Communication
| Pattern | Description |
|---|---|
| Event publishing | Producer fires events; consumers subscribe |
| Command queue | Single consumer processes work item |
| Pros | Loose coupling, buffering, resilience |
| Cons | Eventual consistency, debugging complexity |
3. Implementing Request/Reply Pattern
| Field | Use |
|---|---|
| reply-to | Queue/topic where reply is sent |
| correlation-id | Match reply to request |
| Timeout | Drop request if no reply in N seconds |
| Use case | RPC over messaging (NATS request, RabbitMQ RPC) |
4. Implementing Fire-and-Forget Pattern
| Aspect | Detail |
|---|---|
| Semantics | Send and proceed; no reply expected |
| Delivery | At-least-once via broker ack |
| Idempotency | Required on consumer (dup possible) |
| Use cases | Logging, audit, notifications, telemetry |
5. Handling Service Timeouts
| Layer | Default Timeout |
|---|---|
| Connect | 1–2s |
| Read/socket | 2–5s |
| Overall request | 1–10s (per SLA) |
| Background job | Minutes–hours; explicit per job |
| Rule | Caller timeout > downstream timeout + retries |
6. Implementing Retry Mechanisms
| Strategy | Detail |
|---|---|
| Exponential Backoff | delay = base * 2^attempt |
| Jitter | Random offset prevents thundering herd |
| Max attempts | 3–5 typical |
| Retry-only-on | Idempotent ops + transient errors (5xx, 429, network) |
| Budget | Cap retries as % of total RPS to prevent retry storms |
Example: Resilience4j retry (Java)
RetryConfig cfg = RetryConfig.custom()
.maxAttempts(4)
.intervalFunction(IntervalFunction.ofExponentialRandomBackoff(200, 2.0, 0.5))
.retryOnException(e -> e instanceof IOException)
.build();
Retry retry = Retry.of("payments", cfg);
Payment p = retry.executeSupplier(() -> client.charge(req));7. Managing Connection Pooling
| Param | Guideline |
|---|---|
| Max connections | 2–4× CPU cores per upstream |
| Idle timeout | 30–60s |
| Max lifetime | 10–30 min (refresh DNS, LB) |
| Acquire timeout | Fail-fast if pool exhausted |
| HTTP/2 | Use multiplexing; fewer connections needed |
8. Using Service Mesh
| Capability | Provided By Mesh |
|---|---|
| mTLS | Sidecar handles cert rotation |
| Retries/Timeouts | Configured via CRDs, not code |
| Circuit Breaking | Outlier detection |
| Traffic Splitting | Canary, A/B, mirroring |
| Telemetry | Auto metrics + traces |
9. Handling Partial Failures
| Failure | Strategy |
|---|---|
| Optional dependency down | Graceful degradation (return defaults) |
| Critical dependency down | Fail fast with clear error |
| Slow dependency | Timeout + circuit breaker |
| Partial data | Return what's available + status flag |
10. Implementing Bulkheads
| Type | Example |
|---|---|
| Thread pool | Separate executor per downstream |
| Connection pool | One pool per upstream |
| Process | Dedicated workers per workload |
| Cluster | Tenant-isolated namespaces |
11. Understanding Communication Trade-offs
| Decision Driver | Pick |
|---|---|
| User-facing immediate response | Sync |
| Long-running / fan-out / decoupled | Async |
12. Implementing Correlation ID Propagation
| Header | Description |
|---|---|
| X-Request-ID | Per-request unique ID (UUID) |
| X-Correlation-ID | Business workflow ID across services |
| traceparent | W3C Trace Context (preferred) |
| tracestate | Vendor-specific extensions |
| baggage | Cross-cutting key-value (tenant, user) |