Configuring Logging and Monitoring
1. Setting Up Access Logs
log_format json_combined escape=json
'{"ts":"$time_iso8601",'
'"remote_addr":"$remote_addr",'
'"method":"$request_method",'
'"path":"$uri",'
'"status":$status,'
'"bytes":$body_bytes_sent,'
'"rt":$request_time,'
'"ua":"$http_user_agent",'
'"req_id":"$request_id",'
'"upstream_status":"$upstream_status"}';
access_log /var/log/nginx/access.json json_combined;
2. Configuring Error Logs
| Level | Use |
| debug | Dev only |
| info | Operational events |
| notice | Default prod |
| warn | Recoverable |
| error | Request failed |
| crit/alert/emerg | Page on-call |
3. Implementing Structured Logging
| Field | Example |
ts | ISO 8601 UTC |
level | info/warn/error |
service | api-gateway |
trace_id | W3C trace ID |
span_id | W3C span |
request_id | Per-request UUID |
user_id | Authenticated subject |
route | Matched route name |
4. Using Log Levels
| Event | Level |
| 2xx response | info / not logged |
| 4xx response | info (per-request) or warn (rate) |
| 5xx response | error |
| Cert expiring < 14d | warn |
| Upstream all down | crit |
5. Setting Up Request/Response Logging
Warning: Logging request/response bodies risks PII/secret exposure. Always redact, sample, or disable in prod.
| Strategy | Detail |
| Headers only | Default safe |
| Body sample 1% | Debugging signal |
| Redact fields | password, token, ssn, card |
| Only on errors | 5xx only |
6. Using Correlation IDs
Example: Correlation ID propagation
String requestId = request.getHeader("X-Request-Id");
if (requestId == null) requestId = UUID.randomUUID().toString();
MDC.put("request_id", requestId);
response.setHeader("X-Request-Id", requestId);
// Forward to upstream
upstreamRequest.setHeader("X-Request-Id", requestId);
7. Configuring Log Sampling
| Strategy | Rate |
| All errors | 100% |
| Successful | 1-10% |
| Health checks | 0% (drop) |
| High-value endpoints | 100% |
| Per-trace | Same decision as trace sampler |
8. Implementing Log Filtering
| Filter | Action |
| Health check paths | Drop |
| Bot user agents | Tag, separate stream |
| Secret patterns | Redact sk_*, bearer |
| PII regex | Mask email, card |
9. Setting Up Log Rotation
| Setting | Recommended |
| Rotate by size | 100 MB |
| Rotate by time | Daily |
| Compress old | gzip/zstd |
| Retention | 30-90 days local, 1y archive |
| Reload signal | USR1 (NGINX) |
10. Configuring Audit Logs
| Event | Fields |
| Auth success/fail | user, ip, method, ts |
| Permission change | actor, target, before, after |
| Admin API call | full request body |
| Key rotation | old hash, new hash, actor |
| Storage | WORM (write-once), separate from app logs |