Managing Backend Services
1. Registering Upstream Services
| Field | Description | Example |
|---|
name | Service ID | orders-svc |
protocol | http/https/grpc/tcp | https |
host | DNS or upstream name | orders.internal |
port | TCP port | 8443 |
path | Base prefix on upstream | /api |
Example: Multi-target upstream (Kong)
upstreams:
- name: orders-pool
algorithm: round-robin
targets:
- target: orders-1.internal:8080
weight: 100
- target: orders-2.internal:8080
weight: 100
- target: orders-canary.internal:8080
weight: 10
services:
- name: orders-svc
host: orders-pool # references upstream
port: 8080
2. Configuring Service Health Checks
| Type | How | When |
|---|
| Active | Gateway pings /health | Continuous monitoring |
| Passive | Track real request failures | Always (free) |
| TCP | Open socket check | Non-HTTP backends |
| HTTP | Status code check | Web services |
| gRPC | grpc.health.v1 | gRPC services |
Example: Active health check
healthchecks:
active:
type: http
http_path: /health
timeout: 2
interval: 5 # seconds between checks
healthy:
successes: 2
http_statuses: [200, 204]
unhealthy:
tcp_failures: 3
http_failures: 3
http_statuses: [429, 500, 503]
3. Setting Service Timeouts
| Timeout | Phase | Default |
|---|
connect_timeout | TCP/TLS handshake | 5s |
read_timeout | Receive response | 60s |
write_timeout | Send request body | 60s |
per_try_timeout | Single retry attempt | = read_timeout |
| Metadata | Use |
|---|
tags | Group by env, team, criticality |
labels | K8s-style selector matching |
annotations | Tooling hints (Prom scrape, etc.) |
version | Multi-version routing |
5. Implementing Service Discovery Integration
| Discovery | Mechanism | Example |
|---|
| DNS | SRV/A record lookup | orders.svc.cluster.local |
| Consul | Catalog + health | orders.service.consul |
| Eureka | Spring Cloud registry | REST polling |
| Kubernetes | Endpoints API | Service object |
| Static | Hardcoded list | Targets in config |
6. Configuring Multiple Service Versions
routes:
- name: orders-v2
paths: ["/orders"]
headers:
X-API-Version: ["2"]
service: orders-v2-svc
- name: orders-v1
paths: ["/orders"]
service: orders-v1-svc # default fallback
7. Configuring Retry Policies
| Setting | Description | Default |
|---|
retries | Max attempts | 5 |
retry_on | Conditions: 5xx, gateway-error, reset | 5xx |
per_try_timeout | Single attempt limit | 10s |
backoff | exponential, base 25ms | jittered |
retriable_status_codes | Explicit list | [502,503,504] |
Warning: Never retry non-idempotent methods (POST) unless using idempotency keys.
8. Setting Circuit Breaker Thresholds
| Setting | Description | Typical |
|---|
max_connections | Trip if exceeded | 1024 |
max_pending_requests | Queue cap | 100 |
consecutive_5xx | Eject after N errors | 5 |
base_ejection_time | Open duration | 30s |
max_ejection_percent | Hosts ejected cap | 50% |
9. Configuring Service Fallbacks
Example: Fallback to cached/static response
plugins:
- name: proxy-cache
config:
response_code: [200]
content_type: ["application/json"]
cache_ttl: 300
- name: fallback-response
config:
on_error: true
status: 200
body: '{"items":[],"degraded":true}'
10. Setting Up Blue-Green Deployments
Blue/Green Cutover
- Deploy green stack alongside blue
- Smoke test green via internal route
- Switch gateway upstream from blue → green
- Monitor error rate and latency
- Keep blue warm for instant rollback (30 min)
- Decommission blue after stability window
| Strategy | Risk | Rollback |
|---|
| Blue/Green | Low | Instant (flip route) |
| Canary | Very low | Reduce weight to 0 |
| Rolling | Medium | Re-deploy old |