How does Kafka ensure data ordering guarantees?

Answer

Kafka's ordering guarantees are nuanced. Within a partition: messages are strictly ordered — offset 0 is before offset 1 — always. Across partitions: no ordering guarantee. The fundamental design decision: messages with the same key always go to the same partition (via key hash), ensuring per-key ordering. Producer complications: with max.in.flight.requests.per.connection > 1 and retries enabled, a failed batch can be overtaken by a later batch, breaking in-partition ordering. Fix: enable idempotent producer (enable.idempotence=true) which preserves ordering with up to 5 in-flight requests. Consumer complications: a consumer processing messages from the same partition in parallel threads can process them out of order. Always process one partition serially per consumer thread or use a per-key processing queue. Exactly-once with ordering: Kafka Streams with exactly_once_v2 processes records in order per partition while maintaining exactly-once guarantees.