Handling Errors and Exceptions

1. Understanding Producer Exceptions

Category Description Detail
Retriable Transient, auto-retry RetriableException
Non-Retriable Fatal, no retry Handle/DLQ
Delivery Via callback/Future Async

Example: Callback handling

producer.send(record, (md, ex) -> {
  if (ex != null) handle(ex);
});

2. Handling Timeout Exceptions

Exception Cause Fix
TimeoutException delivery.timeout exceeded Increase/retry
Metadata Timeout Broker unreachable Check network
Buffer Full max.block.ms hit Slow down

Example: Catch timeout

if (ex instanceof TimeoutException) {
  metrics.timeout(); retryLater(record);
}

3. Dealing with Serialization Errors

Aspect Description Detail
SerializationException Bad payload/schema Non-retriable
Producer Fails before send Validate first
Consumer Poison pill Skip/DLQ

Example: Guard serialization

try { producer.send(record); }
catch (SerializationException e) { dlq.send(record); }

4. Handling Authorization Failures

Exception Cause Fix
TopicAuthorizationException No topic ACL Grant access
GroupAuthorizationException No group ACL Add Read
Non-Retriable Won't fix itself Alert ops

Example: Handle auth failure

catch (TopicAuthorizationException e) {
  log.error("Missing ACL for {}", e.unauthorizedTopics());
}

5. Managing Retriable Exceptions

Aspect Description Detail
Auto-Retry Producer retries Within timeout
Idempotence Avoid dup on retry enable
Backoff Spacing between tries retry.backoff.ms

Example: Safe retries

props.put("enable.idempotence", true);
props.put("retries", Integer.MAX_VALUE);

6. Handling Non-Retriable Errors

Aspect Description Detail
Detect Not RetriableException instanceof
Action DLQ or alert No retry
Examples RecordTooLarge, auth Fatal

Example: Branch on retriability

if (ex instanceof RetriableException) retry(r);
else { dlq.send(r); alert(ex); }

7. Understanding Consumer Exceptions

Exception Cause Fix
CommitFailedException Rebalance during commit Retry/handle
WakeupException Intentional shutdown Expected
RecordDeserialization Poison record Skip

Example: Wakeup shutdown

try { while (running) consumer.poll(d); }
catch (WakeupException e) { /* shutdown */ }
finally { consumer.close(); }

8. Handling Offset Out of Range

Config Description Detail
auto.offset.reset earliest/latest/none Recovery
Cause Offset deleted by retention Gap
none Throws exception Manual handle

Example: Reset policy

props.put("auto.offset.reset", "earliest");

9. Dealing with Rebalance Exceptions

Aspect Description Detail
CommitFailed Partitions revoked Commit in listener
Listener onPartitionsRevoked Final commit
Cooperative Reduce disruption Incremental

Example: Commit before revoke

new ConsumerRebalanceListener() {
  public void onPartitionsRevoked(Collection<TopicPartition> p){
    consumer.commitSync();
  }
};

10. Configuring Error Handlers

Aspect Description Detail
UncaughtException Thread crash policy Streams
Global Handler Catch-all logging Centralize
Framework Spring error handlers DefaultErrorHandler

Example: Streams uncaught handler

streams.setUncaughtExceptionHandler(e -> {
  log.error("Fatal", e);
  return StreamThreadExceptionResponse.SHUTDOWN_CLIENT;
});