Question 15 of 50
Why do Partitions exist?
Question
"Why does Kafka split a topic into Partitions? What problem does that solve?"
What the interviewer wants to assess
Whether you understand the motivation behind partitions, not just the definition. This question usually comes right after "what is a partition," testing whether the candidate understands the "why," not just memorized the "what."
Resposta rápida
Partitions exist to solve two problems at once: consumption parallelism (more partitions means more consumers processing in parallel within a Consumer Group) and write scalability (partitions of the same topic can live on different brokers, spreading the load).
Resposta nível Sênior
A topic could, in principle, be a single giant log, but that would limit consumption parallelism to a single sequential stream, no matter how many consumers were added to the group. Partitions solve this by splitting the topic into independent units: within a Consumer Group, each partition is assigned to exactly one consumer at a time, so consumption parallelism scales with the number of partitions. On the write side, partitions of the same topic can be spread across multiple brokers, multiplying the topic's simultaneous write capacity.
In-depth explanation
See "Why partitions exist" in Chapter 4.
Exemplo financeiro
A card transactions topic with a single partition would limit processing to one consumer at a time, even if the team scaled the service to 10 replicas, 9 of them would sit idle. With 10 partitions, all 10 replicas process in genuine parallel.
"Partitions only exist for write performance"
Write performance is half the answer. The other half (just as important and often forgotten) is consumption parallelism: without partitions, adding more consumers to a Consumer Group wouldn't increase processing throughput at all.
Pode vir a seguir
Likely follow-ups: "how would you choose the number of Partitions?" and "what happens if I have more consumers than Partitions?".
Related chapters