Implementing Service Discovery
1. Using DNS-Based Discovery
| Target | Use |
|---|---|
dns:///host:port | Standard DNS A/AAAA |
| SRV records | Optional service+port discovery |
| Kubernetes headless svc | Returns all pod IPs |
2. Implementing Custom Resolver
| Interface | Method |
|---|---|
resolver.Builder | Build, Scheme |
resolver.Resolver | ResolveNow, Close |
cc.UpdateState | Push address list to balancer |
3. Registering Custom Resolver
Example: Register at init
func init() { resolver.Register(&consulBuilder{client: consulCli}) }
// dial: grpc.NewClient("consul:///user-service")4. Using Consul for Discovery
| Step | Detail |
|---|---|
| Register service | Agent HTTP API on startup |
| Health check | gRPC health endpoint |
| Query | Health.Service(name, "", passingOnly, q) |
| Watch | Long-poll with WaitIndex |
5. Using etcd for Discovery
| Mechanism | Detail |
|---|---|
| Put key with lease | /services/user/<id> → ip:port |
| Keep-alive lease | Auto-deregister on crash |
| Watch prefix | Stream membership changes |
6. Using Kubernetes Service Discovery
| Approach | Detail |
|---|---|
| ClusterIP svc | Single VIP; L4 LB — defeats client-side LB |
| Headless svc | DNS returns all pod IPs — enables round_robin |
| xDS / Istio | Control plane pushes endpoints |
7. Implementing Static Resolution
Example: Manual resolver with static list
r := manual.NewBuilderWithScheme("static")
r.InitialState(resolver.State{Addresses: []resolver.Address{
{Addr: "10.0.0.1:50051"}, {Addr: "10.0.0.2:50051"},
}})
conn, _ := grpc.NewClient("static:///user-service",
grpc.WithResolvers(r),
grpc.WithDefaultServiceConfig(`{"loadBalancingConfig":[{"round_robin":{}}]}`),
)8. Updating Resolver State
9. Handling Service Updates
| Concern | Detail |
|---|---|
| Graceful add | Balancer connects new subchannel before using |
| Graceful remove | Drain in-flight on removed subchannel |
| Flapping | Apply hysteresis in resolver |
10. Testing Service Discovery
| Test | Detail |
|---|---|
| Manual resolver | Drive state changes in unit tests |
| Failure injection | Mark subchannels TRANSIENT_FAILURE |
| Integration | Spin up Consul/etcd in docker |