Using Kafka CLI Tools
1. Managing Topics
| Action | Flag | Detail |
|---|---|---|
| Create | --create | + partitions/RF |
| Alter | --alter --partitions | Increase |
| Describe | --describe | Metadata |
Example: kafka-topics
kafka-topics.sh --bootstrap-server localhost:9092 \
--create --topic events --partitions 6 --replication-factor 3
2. Working with Consumer Groups
| Action | Flag | Detail |
|---|---|---|
| List | --list | All groups |
| Describe | --describe --group | Lag/members |
| Reset | --reset-offsets | Move offset |
Example: kafka-consumer-groups
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
--describe --group order-svc
3. Managing ACLs
| Action | Flag | Detail |
|---|---|---|
| Add | --add | Grant |
| List | --list | View |
| Remove | --remove | Revoke |
Example: kafka-acls
kafka-acls.sh --bootstrap-server localhost:9092 --add \
--allow-principal User:alice --operation Read --topic orders
4. Managing Configs
| Action | Flag | Detail |
|---|---|---|
| Describe | --describe | Current |
| Add | --add-config | Set |
| Delete | --delete-config | Remove |
Example: kafka-configs
kafka-configs.sh --bootstrap-server localhost:9092 --alter \
--entity-type topics --entity-name orders \
--add-config retention.ms=86400000
5. Reassigning Partitions
| Step | Flag | Detail |
|---|---|---|
| Generate | --generate | Proposed plan |
| Execute | --execute | Apply |
| Verify | --verify | Progress |
Example: Reassign partitions
kafka-reassign-partitions.sh --bootstrap-server localhost:9092 \
--reassignment-json-file plan.json --execute
6. Preferred Leader Election
| Aspect | Description | Detail |
|---|---|---|
| Preferred | First replica in list | Balanced |
| --election-type | PREFERRED | Restore balance |
| Auto | auto.leader.rebalance.enable | Default on |
Example: Trigger election
kafka-leader-election.sh --bootstrap-server localhost:9092 \
--election-type PREFERRED --all-topic-partitions
7. Verifying Consumer Group
| Column | Description | Detail |
|---|---|---|
| CURRENT-OFFSET | Committed position | Per partition |
| LOG-END-OFFSET | Latest available | HW |
| LAG | End − current | Backlog |
Example: Check lag
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
--describe --group order-svc | awk '{print $1,$2,$6}'
8. Verifying Producer
| Tool | Description | Detail |
|---|---|---|
| producer-perf-test | Throughput benchmark | Synthetic |
| --num-records | Total messages | Count |
| --throughput | Rate limit | -1 = max |
Example: Producer perf test
kafka-producer-perf-test.sh --topic perf --num-records 100000 \
--record-size 1000 --throughput -1 \
--producer-props bootstrap.servers=localhost:9092
9. Dumping Log Segments
| Flag | Description | Detail |
|---|---|---|
| --files | Segment .log files | Input |
| --print-data-log | Show record payloads | Verbose |
| --deep-iteration | Per-record detail | Batches |
Example: Dump log
kafka-dump-log.sh --files /data/orders-0/00000000000000000000.log \
--print-data-log
10. Mirroring Data
| Tool | Description | Detail |
|---|---|---|
| connect-mirror-maker | MirrorMaker 2.0 | Cross-cluster |
| Config File | Clusters + flows | Properties |
| Use | Replication/DR | Geo |
11. Running Performance Tests
| Tool | Description | Detail |
|---|---|---|
| consumer-perf-test | Consumer throughput | MB/s |
| --messages | Count to consume | Limit |
| Output | Rate + latency | CSV |