How does Kafka handle back pressure?
Answer
Kafka's pull-based consumer model is its primary back pressure mechanism — consumers fetch messages at their own pace, so a slow consumer does not overwhelm itself; it simply accumulates lag. This is fundamentally different from push-based systems where a fast producer can overwhelm a slow consumer. Consumer-side controls: max.poll.records: limit messages per poll (default 500). fetch.min.bytes/fetch.max.wait.ms: control polling frequency. max.partition.fetch.bytes: limit data per partition per fetch. Producer-side controls: producers buffer messages and apply linger.ms and batch.size. If the producer's send buffer fills up, send() blocks until space is available (controlled by max.block.ms). For Kafka Streams: buffered.records.per.partition limits in-memory buffering. The key insight: Kafka's durability (disk-backed buffer) means the back pressure mechanism is simply the natural lag between producer and consumer, which is visible and measurable.
Previous
What is the difference between Kafka Streams and Apache Flink?
Next
How do you tune Kafka for ultra-low latency?