What is the difference between message queues and event streaming platforms?

Why Interviewers Ask This

Candidates at the intermediate level are expected to not only know this concept but explain the trade-offs involved. Interviewers use this question to see if you can reason about design decisions, not just recall facts.

Answer

Message queues (RabbitMQ, AWS SQS): designed for task distribution — a message is consumed by ONE consumer and then deleted from the queue. Pull or push delivery. Consumers compete for messages (competing consumer pattern). Messages are typically short-lived (consumed and gone). Use for: task queues (send one email per event), request-response patterns, work distribution among workers. Event streaming platforms (Apache Kafka, AWS Kinesis): designed as a distributed, persistent log — events are retained for a configured retention period (days to forever). Multiple independent consumer groups each read ALL events from their own position (offset) in the log — events are not deleted after consumption. Producers write to the end of the log; consumers read from any offset. Use for: event sourcing, audit logs, real-time analytics, multiple systems consuming the same event stream, event replay, stream processing (Kafka Streams, Apache Flink). Key differences: message queue — messages disappear after consumption, consumers are equivalent (one processes each message); event log — events persist, multiple consumers independently process all events. Kafka mechanics: topics split into partitions (parallel processing); each partition is an ordered, immutable sequence of records; consumers track their offset per partition; consumer groups — each partition is consumed by one consumer in the group (scalability); replication for durability. When to use each: need task distribution with one processor per task → message queue; need multiple systems to react to the same events, event replay, audit trail → Kafka.

Common Mistake

Candidates often give textbook answers here. Interviewers are more impressed when you relate the concept to a specific problem you solved in a real System Design project.