Part I — Fundamentals
Kafka versus other tools
A technical comparison between Kafka, RabbitMQ, AWS SQS, AWS SNS, and Google Pub/Sub.
On this page
One of the most common interview questions is comparing Kafka to some other messaging tool the candidate has already used. This isn't naming pedantry — each tool was designed for a different usage model, and choosing wrong has a real cost in production.
The axis that actually matters: work queue vs. event log
Before comparing features, it's worth splitting these tools into two families:
- Work queues (RabbitMQ, AWS SQS): each message should be processed by exactly one consumer; once processed (ack), it normally disappears.
- Event log / pub-sub with retention (Kafka, and — with caveats — AWS SNS + Google Pub/Sub): the same event can be read by multiple independent consumers, and can stay available after being read.
Ignoring this difference is the most common cause of choosing the wrong tool.
Kafka vs. RabbitMQ
Kafka vs. RabbitMQ
| Criterion | Kafka | RabbitMQ |
|---|---|---|
| Model | Distributed, partitioned log | Message queue (with exchanges/routing) |
| Retention | Configurable (time/size), message persists after consumption | Message disappears after ack (unless configured otherwise) |
| Replay | Native — resetting the offset rereads history | Not native — requires manual resend |
| Ordering | Guaranteed per partition | Guaranteed per queue (FIFO), simpler to reason about in a single queue |
| Multiple consumers of the same data | Natural, via separate Consumer Groups | Requires explicit fan-out (a fanout exchange) |
| Throughput at high scale | Very high, designed for volume | High, but with more routing overhead per message |
| Complex routing (rules, priority) | Limited — routing is by topic/partition/key | Rich — exchanges, routing keys, message priority |
| Operational curve | Higher (cluster, partitions, retention) | Lower for simple cases |
RabbitMQ shines when the requirement is sophisticated routing of work messages — priorities, fine-grained dead lettering, multiple exchange patterns. Kafka shines when the requirement is an event history shared by multiple consumers, with a need for replay.
Kafka vs. AWS SQS
Kafka vs. AWS SQS
| Criterion | Kafka | AWS SQS |
|---|---|---|
| Model | Distributed log | Managed queue (Standard or FIFO) |
| Retention after consumption | Yes (up to the topic's retention limit) | No — message is removed after ack (or expires) |
| Replay | Native | No — only via DLQ redrive (reprocessing failed messages, not full history) |
| Ordering | Per partition | FIFO guarantees order per message group; Standard guarantees no order |
| Operations | You manage the cluster (or use a managed service like MSK/Confluent Cloud) | Fully managed by AWS, zero infrastructure |
| Independent consumer scaling | Native (multiple Consumer Groups) | Requires SNS + multiple queues for fan-out |
| Throughput | Very high | High, with per-queue throughput limits (Standard scales better than FIFO) |
SQS FIFO is not Kafka
SQS FIFO guarantees order within a message group and avoids duplication — but it's still a queue: the message disappears from the queue once read. It solves ordering, not retention/replay, and not multiple independent consumers of the same data without duplicating the message via SNS.
Kafka vs. AWS SNS
SNS is a pure pub/sub service (fan-out): a published message is delivered to every subscriber (SQS queues, HTTP endpoints, Lambda). It's common to find the SNS + SQS pattern: SNS distributes, each SQS queue consumes independently — this partially reproduces what Kafka does natively with Consumer Groups, but without long-term retention or history replay: once delivered and processed, the message can't be reread from SNS.
Kafka vs. Google Pub/Sub
Kafka vs. Google Pub/Sub
| Criterion | Kafka | Google Pub/Sub |
|---|---|---|
| Model | Partitioned log, managed by you (or a managed service) | Fully managed pub/sub |
| Retention | Configurable, typically days | Configurable, up to 31 days (already-acknowledged messages aren't retained the same way by default) |
| Replay | Native via offset | Via seek, supported, but with different semantics than per-partition offset |
| Ordering | Per partition (key) | Via ordering key, optional and with a throughput cost |
| Operations | Requires cluster management (or a managed service) | Zero infrastructure |
Google's Pub/Sub is operationally the closest thing to "Kafka without operating Kafka," but the partition model and the client ecosystem (especially for Java/Spring) still favor Kafka for teams that already have that stack.
A practical decision criterion
Dica de entrevista
When asked to compare, don't memorize the table — explain the decision criterion: "if I need multiple independent services to read the same event, with the possibility of replay, I think Kafka; if I need to distribute work among workers, with rich routing and low retention needs, I think RabbitMQ or SQS." This shows reasoning, not memorization.
Choosing the right tool for each part of the payment flow
In a real payments system, it's common to use both families at the same time: Kafka for the "payment approved" domain event (which antifraud, accounting, and notifications consume independently and may need to reprocess), and an SQS queue for the internal work of "send confirmation email" (a task that should be processed exactly once by a worker, with no need for replay).
Resumo
RabbitMQ and SQS model work queues — a processed message disappears. Kafka models a retained event log, read by multiple independent consumers, with native replay. SNS solves fan-out, but not long-term retention. Google's Pub/Sub is operationally the closest thing to Kafka "without operating a cluster," but with different ordering and retention semantics. The right choice depends on what you need to do with the message after it's first read.
Pode vir a seguir
Also prepare for: "would you use Kafka for everything?", "how would you do fan-out without Kafka?", and "what would happen if you used SQS for a case that needs replay?".