Skip to main content
Kafka
pten

Question 26 of 50

What happens when a Consumer goes down?

Mid-levelSenior

Question

"What happens when a Consumer in a Consumer Group goes down unexpectedly?"

What the interviewer wants to assess

Practically guaranteed in any interview about Kafka in production. Tests whether the candidate knows the failure-detection and redistribution mechanism, not just the end result ("another consumer takes over").

Resposta rápida

The group coordinator detects the missing heartbeat within the configured session.timeout.ms, declares the consumer dead, and triggers a rebalance — the partitions that were assigned to it are redistributed among the group's remaining consumers.

Resposta nível Sênior

Each consumer sends periodic heartbeats to the group coordinator (one of the brokers). If the coordinator doesn't receive a heartbeat within session.timeout.ms, it considers the consumer dead and starts a rebalance, redistributing the orphaned partitions among the active consumers. A subtle point: if processing a message takes longer than max.poll.interval.ms, the consumer is considered dead even if the heartbeat thread keeps responding — because the poll loop, not the heartbeat alone, is the real signal that the consumer is processing normally. This is a common cause of "false" rebalances, where a healthy but slow consumer gets kicked out of the group.

In-depth explanation

See "What triggers a rebalance" in Chapter 6.

Exemplo financeiro

If one of the notificacao-service's 4 replicas goes down due to an out-of-memory error, its partitions are redistributed among the 3 remaining replicas within seconds — with no manual intervention, but with a brief processing pause during the transition.

"A consumer is only considered dead if the process actually crashes"

No — a consumer that's alive but takes too long to process a message (exceeding max.poll.interval.ms) also gets kicked out of the group, even with no real infrastructure failure at all.

Pode vir a seguir

Likely follow-ups: "what is max.poll.interval.ms and why does it cause rebalances?" and "what is Offset?".

Related chapters