Question 24 of 50
What happens when there are more Partitions than consumers?
Question
"What happens if a Consumer Group has fewer consumer instances than the topic has Partitions?"
What the interviewer wants to assess
The counterpoint to the previous question — tests whether the candidate understands the opposite scenario and its load-balancing implications, not just the "ideal" 1:1 case.
Resposta rápida
Some consumers receive more than one partition. Processing stays parallel — every partition still has exactly one consumer assigned — but becomes unbalanced, since consumers with multiple partitions process more load than the others.
Resposta nível Sênior
Kafka distributes the remaining partitions among the available consumers as evenly as possible (depending on the configured assignor strategy), but if the number of partitions isn't a multiple of the number of consumers, the distribution is never perfectly equal — one consumer might get 3 partitions and another 2, for example. That's not a problem by itself, but it means the consumer with more partitions assigned tends to accumulate more consumer lag (Chapter 15) if volume per partition is similar across them, because it has to process twice or three times as many messages in the same time window.
In-depth explanation
See "What happens with different numbers of consumers" in Chapter 6.
Exemplo financeiro
A topic with 5 partitions and a Consumer Group with 2 replicas results in one replica processing 3 partitions and the other processing 2 — still parallel, but with one replica absorbing 50% more load than the other.
"This is a problem I need to fix immediately"
Not always. It's a valid, functional distribution — the real warning sign is when that imbalance turns into disproportionate, sustained consumer lag on one of the instances, a sign that the group's total parallelism is insufficient for the volume, not just poorly distributed.
Pode vir a seguir
Likely follow-ups: "what is Rebalance?" and "what happens when a Consumer goes down?".
Related chapters