Managing Service Discovery
1. Understanding Service Registry Pattern
| Component | Role |
|---|---|
| Registry | Stores instance → endpoint mappings |
| Registrar | Service registers self / sidecar registers |
| Resolver | Client looks up endpoints |
| Health checker | Removes unhealthy instances |
| Examples | Consul, Eureka, etcd, ZooKeeper, Kubernetes API |
2. Implementing Client-Side Discovery
| Pros | Cons |
|---|---|
| No extra hop | Logic per language |
| Smart load balancing | Larger client footprint |
| Examples: Eureka + Ribbon, gRPC name resolver | Coupling to registry |
3. Implementing Server-Side Discovery
| Pros | Cons |
|---|---|
| Polyglot clients | Extra hop, latency |
| Centralized policies | LB is bottleneck/SPOF |
| Examples: AWS ALB, Kubernetes Service, Envoy | — |
4. Implementing Health Checks
| Type | Endpoint | Use |
|---|---|---|
| Liveness | /healthz | Restart if failing |
| Readiness | /ready | Remove from LB pool |
| Startup | /startup | Slow boot grace period |
| Deep health | Includes dependencies | Beware cascades |
| Shallow health | Process up only | Default for liveness |
5. Implementing Heartbeat Mechanisms
| Pattern | Detail |
|---|---|
| Push heartbeat | Service POSTs to registry every N seconds |
| Pull (probe) | Registry pings service health endpoint |
| Lease renewal | etcd/ZooKeeper TTL refresh |
| Phi accrual | Probabilistic suspicion (Akka, Cassandra) |
6. Working with DNS-Based Discovery
| Record | Use |
|---|---|
| A / AAAA | Hostname → IP |
| SRV | Service + port + priority + weight |
| CNAME | Alias for service endpoint |
| Headless service (k8s) | Returns all pod IPs |
| TTL pitfalls | Stale cache delays failover |
7. Implementing Service Registration
| Pattern | Detail |
|---|---|
| Self-registration | Service registers on startup |
| Third-party (registrar) | Sidecar / orchestrator registers |
| Kubernetes | Endpoints auto-managed by controller |
8. Handling Service Deregistration
| Trigger | Action |
|---|---|
| Graceful shutdown hook | Service deregisters |
| Lease expiration | Registry removes after TTL |
| Health check failure | Mark unhealthy → remove |
| Drain period | Stop accepting new; finish in-flight |
9. Implementing Service Metadata Management
| Metadata | Use |
|---|---|
| Version | Canary, traffic split |
| Region/AZ | Locality routing |
| Capacity / weight | Weighted LB |
| Tags | Filtering (env=prod) |
10. Understanding Service Discovery Trade-offs
| Trade | Detail |
|---|---|
| Strong vs eventual consistency | etcd vs gossip |
| Push vs pull | Latency vs simplicity |
| Centralized vs P2P | SPOF vs complex |
| DNS vs API | Universal vs feature-rich |