Configuring Quotas

1. Understanding Quotas Concept

Type Description Detail
Network Byte rate limit producer/consumer
Request CPU/IO % request_percentage
Enforcement Throttle via delay Backpressure

Example: Quota throttling

over-quota -> broker delays response -> client slows down

2. Setting Producer Quotas

Config Description Detail
producer_byte_rate Max produce B/s Per quota entity
Throttle Delay produce response Smooth
Metric produce-throttle-time Monitor

Example: Producer quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'producer_byte_rate=1048576' \
  --entity-type clients --entity-name app1

3. Configuring Consumer Quotas

Config Description Detail
consumer_byte_rate Max fetch B/s Per entity
Throttle Delay fetch response Limit
Metric fetch-throttle-time Monitor

Example: Consumer quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'consumer_byte_rate=2097152' \
  --entity-type clients --entity-name app1

4. Setting Request Quotas

Config Description Detail
request_percentage % of broker thread time Per entity
100 × cores Total available Scale
Protects Broker CPU saturation Fairness

Example: Request quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'request_percentage=200' \
  --entity-type users --entity-name analytics

5. Configuring Client-Level Quotas

Entity Description Detail
clients By client.id App-level
--entity-name Specific client id Targeted
Granular Per application Isolation

Example: Client.id quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'producer_byte_rate=524288' \
  --entity-type clients --entity-name batch-loader

6. Setting User-Level Quotas

Entity Description Detail
users By authenticated principal Identity-level
user + client Combined entity Finest grain
Precedence Specific over default Hierarchy

Example: User quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'consumer_byte_rate=1048576' \
  --entity-type users --entity-name alice

7. Using Default Quotas

Aspect Description Detail
--entity-default Applies to all Fallback
Override Named entity wins Specific
Use Baseline protection Cluster-wide

Example: Default user quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --add-config 'producer_byte_rate=1048576' \
  --entity-type users --entity-default

8. Configuring Quota Window

Config Description Default
quota.window.
size.seconds		 Per-sample window 1
quota.window.num Number of samples 11
Effect Smoothing period Burst tolerance

Example: Quota window

quota.window.size.seconds=1
quota.window.num=11

9. Monitoring Quota Violations

Metric Description Detail
throttle-time Delay applied Client metric
byte-rate Current usage Per entity
JMX Broker quota metrics Per client

Example: Throttle metric

kafka.producer:type=producer-metrics,client-id=app1
  -> produce-throttle-time-avg

10. Dynamic Quota Configuration

Aspect Description Detail
Dynamic No restart needed Live
Delete --delete-config Remove quota
Describe View current quotas --describe

Example: Remove a quota

kafka-configs.sh --bootstrap-server localhost:9092 --alter \
  --delete-config 'producer_byte_rate' \
  --entity-type clients --entity-name app1

Related Cheat Sheets

Kafka

Installing and Configuring Kafka

Kafka

Implementing Stream Joins

Kafka

Working with Partitions

Kafka

Understanding Message Compression