Question 15 of 50
Why do Partitions exist?
Question
"Why does Kafka split a topic into Partitions? What problem does that solve?"
What the interviewer wants to assess
Whether you understand the motivation behind partitions, not just the definition — this question usually comes right after "what is a partition," testing whether the candidate understands the "why," not just memorized the "what."
Resposta rápida
Partitions exist to solve two problems at once: consumption parallelism (more partitions means more consumers processing in parallel within a Consumer Group) and write scalability (partitions of the same topic can live on different brokers, spreading the load).
Resposta nível Sênior
A topic could, in principle, be a single giant log — but that would limit consumption parallelism to a single sequential stream, no matter how many consumers were added to the group. Partitions solve this by splitting the topic into independent units: within a Consumer Group, each partition is assigned to exactly one consumer at a time, so consumption parallelism scales with the number of partitions. On the write side, partitions of the same topic can be spread across multiple brokers, multiplying the topic's simultaneous write capacity.
In-depth explanation
See "Why partitions exist" in Chapter 4.
Exemplo financeiro
A card transactions topic with a single partition would limit processing to one consumer at a time, even if the team scaled the service to 10 replicas — 9 of them would sit idle. With 10 partitions, all 10 replicas process in genuine parallel.
"Partitions only exist for write performance"
Write performance is half the answer. The other half — just as important and often forgotten — is consumption parallelism: without partitions, adding more consumers to a Consumer Group wouldn't increase processing throughput at all.
Pode vir a seguir
Likely follow-ups: "how would you choose the number of Partitions?" and "what happens if I have more consumers than Partitions?".
Related chapters