Question 47 of 50
What is Consumer Lag?
Question
"What is Consumer Lag, and why is it the most important metric for monitoring a Kafka consumer?"
What the interviewer wants to assess
Whether you connect the metric to the business symptom it anticipates — not just the isolated technical definition.
Resposta rápida
Consumer Lag is the difference between the latest offset already produced on a partition (log end offset) and the offset committed by the Consumer Group — it represents how many messages already exist but haven't been processed yet.
Resposta nível Sênior
Zero lag (or fluctuating close to zero with normal traffic) indicates a healthy consumer, keeping up with production's pace. The real warning sign isn't a momentary lag spike during a traffic surge — that's expected and usually resolves on its own — but lag that grows consistently over time, which indicates consumption throughput is structurally lower than production throughput. It's the most important metric precisely because it anticipates a business problem (processing delay) before the end user notices it — unlike an error rate, which only shows up after something has already gone wrong.
In-depth explanation
See "Consumer Lag: the most important vital sign" in Chapter 15.
Exemplo financeiro
An alert configured in Grafana fires if the saldo-service's consumer lag on the pix.recebido topic
exceeds 1000 messages for more than 2 minutes — because, for this flow, even a few seconds of delay between
receiving the PIX and crediting the balance is already noticeable to the customer.
"Any lag greater than zero is already a problem"
Momentary lag during traffic spikes is normal and expected. The real problem is the trend: lag that doesn't drop back down after the spike passes, or that grows continuously — not an isolated instantaneous value.
Pode vir a seguir
Likely follow-ups: "how would you monitor Kafka?" and "what would you do if consumer lag were growing continuously?".
Related chapters