Configuring Consumer Settings

1. Setting Fetch Size

Config Description Default
fetch.min.bytes Min data before broker responds 1
fetch.max.bytes Max bytes per fetch 52428800 (50MB)
Higher min Better batching, more latency Throughput tuning

Example: Batch-oriented fetch

props.put("fetch.min.bytes", 65536);
props.put("fetch.max.bytes", 104857600);

2. Configuring Fetch Wait Time

Config Description Default
fetch.max.wait.ms Max wait to satisfy fetch.min.bytes 500
Latency Cap Returns even if min not met Upper bound
Tradeoff Lower = lower latency More requests

Example: Reduce fetch latency

props.put("fetch.max.wait.ms", 100);

3. Setting Max Poll Records

Config Description Default
max.poll.records Records returned per poll() 500
Processing Time Lower if per-record work is heavy Avoid timeout
Throughput Higher batches more per loop Balance

Example: Limit poll batch

props.put("max.poll.records", 100);

4. Setting Max Partition Fetch Bytes

Config Description Default
max.partition.
fetch.bytes		 Max bytes per partition 1048576 (1MB)
Large Messages Must exceed largest record Else stalls
Memory × assigned partitions = total Plan heap

Example: Increase per-partition fetch

props.put("max.partition.fetch.bytes", 5242880); // 5MB

5. Configuring Partition Assignment Strategy

Strategy Behavior Detail
RangeAssignor Per-topic ranges Default
RoundRobinAssignor Even spread Balanced
CooperativeSticky Incremental rebalance NEW Less disruption

Example: Set strategy

props.put("partition.assignment.strategy",
    "org.apache.kafka.clients.consumer.CooperativeStickyAssignor");

6. Setting Auto Offset Reset

Value Behavior Use Case
earliest Start at oldest offset Full processing
latest Only new records Live tailing
none Throw if no offset Strict control

Example: Process from beginning

props.put("auto.offset.reset", "earliest");

7. Configuring Isolation Level

Level Behavior Use Case
read_uncommitted See all records Default
read_committed Skip aborted transactions Exactly-once
LSO Last stable offset boundary Tx-aware

Example: Read committed only

props.put("isolation.level", "read_committed");

8. Setting Session Timeout

Config Description Default
session.timeout.ms Time before member declared dead 45000
Bounds Within group.min/max bounds Broker-enforced
Tradeoff Lower = faster detection More rebalances

Example: Tune session timeout

props.put("session.timeout.ms", 30000);

9. Setting Heartbeat Interval

Config Description Default
heartbeat.interval.ms Heartbeat send frequency 3000
Rule ≤ 1/3 of session.timeout.ms Best practice
Background Sent by heartbeat thread Independent of poll

Example: Heartbeat config

props.put("heartbeat.interval.ms", 10000); // ~1/3 of session.timeout

10. Configuring Max Poll Interval

Config Description Default
max.poll.interval.ms Max gap between poll() calls 300000
Exceeded Member leaves group Triggers rebalance
Slow Processing Raise for heavy work Or reduce records

Example: Allow longer processing

props.put("max.poll.interval.ms", 600000); // 10 min

11. Configuring Check CRCs

Config Description Default
check.crcs Validate record checksums true
Disable Slight CPU gain, no integrity check Rarely
Detects On-disk/network corruption Safety

Example: CRC check setting

props.put("check.crcs", "true"); // keep enabled for integrity

12. Setting Client ID

Config Description Detail
client.id Logical client name For logging/metrics
Quotas Used to apply client quotas Rate limiting
Tracing Appears in broker logs Debugging

Example: Set client id

props.put("client.id", "billing-consumer-1");

Related Cheat Sheets

Kafka

Installing and Configuring Kafka

Kafka

Understanding Message Compression

Kafka

Using Schema Registry

Kafka

Optimizing Consumer Performance