Managing Cluster Metadata
1. Understanding Metadata Topics
| Topic | Purpose | Detail |
|---|---|---|
| __cluster_metadata | KRaft metadata log | Single partition |
| __consumer_offsets | Committed offsets | 50 partitions |
| __transaction_state | Tx coordinator state | Compacted |
Example: List internal topics
kafka-topics.sh --bootstrap-server localhost:9092 \
--list --exclude-internal=false | grep '^__'2. Viewing Cluster Metadata
| Tool | Description | Detail |
|---|---|---|
| kafka-metadata-shell | Browse metadata snapshot | Filesystem-like |
| ls / cat | Navigate metadata tree | Interactive |
| kafka-metadata- quorum |
Quorum status | Replication state |
Example: Metadata shell
kafka-metadata-shell.sh --snapshot $LOGDIR/__cluster_metadata-0/*.log
>> ls /topics3. Describing Cluster Configuration
| Command | Description | Detail |
|---|---|---|
| --describe | Show cluster-level config | entity-type |
| describeCluster | Admin API node list | controller id |
| Cluster ID | Unique cluster identifier | UUID |
Example: Describe cluster config
kafka-configs.sh --bootstrap-server localhost:9092 \
--entity-type brokers --entity-default --describe4. Managing Broker Configuration
| Command | Description | Detail |
|---|---|---|
| --alter --add-config | Dynamic broker config | No restart |
| --entity-default | Cluster-wide default | All brokers |
| --entity-name | Per-broker override | broker id |
Example: Dynamic broker change
kafka-configs.sh --bootstrap-server localhost:9092 \
--entity-type brokers --entity-name 1 \
--alter --add-config log.cleaner.threads=25. Understanding Controller Role
| Responsibility | Description | Detail |
|---|---|---|
| Leader Election | Assign partition leaders | On failure |
| Metadata Writes | Topic/partition changes | Single writer |
| ISR Management | Track in-sync replicas | Durability |
Example: Find active controller
kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status6. Monitoring Controller Metrics
| Metric | Description | Healthy |
|---|---|---|
| ActiveControllerCount | Should be 1 cluster-wide | =1 |
| LastAppliedRecordOffset | Metadata apply progress | Advancing |
| MetadataErrorCount | Apply errors | 0 |
7. Handling Metadata Updates
| Aspect | Description | Detail |
|---|---|---|
| Propagation | Brokers fetch metadata delta | Incremental |
| Snapshots | Compact metadata log | Faster recovery |
| Lag | Broker behind active controller | Monitor |
Example: Replication lag
kafka-metadata-quorum.sh --bootstrap-server localhost:9092 \
describe --replication8. Configuring Metadata Max Age
| Config | Description | Default |
|---|---|---|
| metadata.max.age.ms | Client metadata refresh | 300000 |
| Lower | Faster topology awareness | More requests |
| Forced | Refresh on errors | Auto |
9. Understanding Partition Metadata
| Field | Description | Detail |
|---|---|---|
| Leader | Broker serving reads/writes | Per partition |
| Replicas | Assigned broker list | Order = preferred |
| ISR | In-sync replica set | Subset |
Example: Describe partition metadata
kafka-topics.sh --bootstrap-server localhost:9092 \
--describe --topic orders10. Debugging Metadata Issues
| Symptom | Likely Cause | Action |
|---|---|---|
| No active controller | Quorum lost | Check voters |
| Stale leaders | Broker metadata lag | Inspect logs |
| Unknown topic | Metadata not propagated | Refresh |