The book
Organized into progressive parts, from fundamentals to production observability. Each chapter can be read independently, but the order below is recommended for beginners.
Part I — Fundamentals
Part II — Architecture
- Internal architecture
Producer, Broker, Cluster, Topic, Partition, Offset, Consumer, and Consumer Group — the pieces that make up Kafka.
- Partitions and ordering
Why partitions exist, how the key determines ordering, and why Kafka doesn't guarantee global order.
- Brokers, leaders, and replication
Replication Factor, Leader, Follower, ISR, and what happens when a broker goes down.
- Consumer Groups and Rebalance
How a Consumer Group splits partitions among consumers, what happens when that split changes, and the real cost of a rebalance.
Part III — Consumption and reprocessing
- Offset and Commit
How a consumer marks how far it has read, the difference between auto commit and manual commit, and the risks of each approach.
- Retention and Replay
How long Kafka keeps an event, and how to use that retention to reprocess history with replay.
- Retry and DLQ
How to tell a transient error from a permanent one, and why Kafka has no native DLQ like SQS does.
Part IV — Reliability
- Delivery guarantees
At Most Once, At Least Once, and Exactly Once — what each guarantee actually promises, and where exactly-once stops applying.
- Idempotency
Why consumers need to be idempotent, and how to implement it with eventId, a uniqueness constraint, and atomicity.
- Transactions and the Outbox Pattern
The dual write problem between a database and Kafka, and how the transactional outbox solves it with guaranteed consistency.
Part V — Java and Spring Boot
- Producer with Spring Kafka
KafkaTemplate in practice — topic, key, value, serialization, callbacks, headers, and correlationId.
- Consumer with Spring Kafka
@KafkaListener in practice — groupId, concurrency, Acknowledgment, deserialization, and exception handling with retry and DLQ.
- Observability
Consumer lag, throughput, structured logs, correlationId, metrics, and tracing — how to know a Kafka pipeline is healthy.