Configuring Broker Settings
1. Setting Broker ID
| Config | Description | Detail |
|---|---|---|
| node.id | KRaft unique identifier | Integer |
| broker.id | Legacy ZK identifier | LEGACY |
| Uniqueness | Must differ per broker | Cluster-wide |
2. Configuring Listeners
| Config | Description | Example |
|---|---|---|
| listeners | Bind endpoints | PLAINTEXT://:9092 |
| advertised.listeners | Client-facing endpoints | host:9092 |
| inter.broker. listener.name |
Internal traffic listener | INTERNAL |
Example: Listener config
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://broker3.example.com:90923. Setting Log Directories
| Config | Description | Tip |
|---|---|---|
| log.dirs | Data directories | One per disk |
| num.recovery. threads.per.data.dir |
Startup recovery parallelism | Faster boot |
| Balance | Partitions spread by free space | Auto |
4. Configuring Number of Threads
| Config | Description | Default |
|---|---|---|
| num.network.threads | Handle network requests | 3 |
| num.io.threads | Disk IO request handlers | 8 |
| background.threads | Misc background tasks | 10 |
5. Setting Socket Buffer Sizes
| Config | Description | Default |
|---|---|---|
| socket.send. buffer.bytes |
TCP send buffer | 102400 |
| socket.receive. buffer.bytes |
TCP receive buffer | 102400 |
| socket.request. max.bytes |
Max request size | 104857600 |
6. Configuring Max Message Size
| Config | Description | Default |
|---|---|---|
| message.max.bytes | Max record batch size | 1048588 |
| replica.fetch. max.bytes |
Must ≥ message.max.bytes | Align |
| Client | max.request.size matched | Both sides |
7. Setting Default Replication Factor
| Config | Description | Prod Value |
|---|---|---|
| default.replication.factor | Auto-created topics | 3 |
| offsets.topic. replication.factor |
__consumer_offsets RF | 3 |
| transaction.state.log. replication.factor |
Tx log RF | 3 |
Example: Replication defaults
default.replication.factor=3
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=38. Configuring Min ISR
| Config | Description | Recommended |
|---|---|---|
| min.insync.replicas | Broker default min ISR | 2 |
| With acks=all | Enforces durability | Pair |
| Override | Per-topic config | Critical topics |
9. Setting Unclean Leader Election
| Config | Description | Default |
|---|---|---|
| unclean.leader. election.enable |
Allow out-of-sync leader | false |
| false | No data loss, may halt | Safe |
| true | Availability over durability | Risky |
10. Configuring Log Retention
| Config | Description | Default |
|---|---|---|
| log.retention.hours | Default retention | 168 (7d) |
| log.retention.bytes | Per-partition size cap | -1 |
| log.retention. check.interval.ms |
Cleanup scan interval | 300000 |
11. Setting Log Segment Size
| Config | Description | Default |
|---|---|---|
| log.segment.bytes | Segment file size | 1073741824 |
| log.roll.hours | Time-based roll | 168 |
| log.index.size. max.bytes |
Max index size | 10485760 |
12. Configuring Log Cleanup
| Config | Description | Default |
|---|---|---|
| log.cleanup.policy | delete / compact | delete |
| log.cleaner.enable | Enable compaction | true |
| log.cleaner.threads | Cleaner threads | 1 |