Question 47 of 50
What is Consumer Lag?
Question
"What is Consumer Lag, and why is it the most important metric for monitoring a Kafka consumer?"
What the interviewer wants to assess
Whether you connect the metric to the business symptom it anticipates, not just the isolated technical definition.
Resposta rápida
Consumer Lag is the difference between the latest offset already produced on a partition (log end offset) and the offset committed by the Consumer Group: it represents how many messages already exist but haven't been processed yet.
Resposta nível Sênior
Zero lag (or fluctuating close to zero with normal traffic) indicates a healthy consumer, keeping up with production's pace. The real warning sign isn't a momentary lag spike during a traffic surge (that's expected and usually resolves on its own), but lag that grows consistently over time, which indicates consumption throughput is structurally lower than production throughput. It's the most important metric precisely because it anticipates a business problem (processing delay) before the end user notices it, unlike an error rate, which only shows up after something has already gone wrong.
In-depth explanation
See "Consumer Lag: the most important vital sign" in Chapter 15.
Exemplo financeiro
An alert configured in Grafana fires if the saldo-service's consumer lag on the pix.recebido topic
exceeds 1000 messages for more than 2 minutes: for this flow, even a few seconds of delay between
receiving the PIX and crediting the balance is already noticeable to the customer.
"Any lag greater than zero is already a problem"
Momentary lag during traffic spikes is normal and expected. The real problem is the trend: lag that doesn't drop back down after the spike passes, or that grows continuously, not an isolated instantaneous value.
Pode vir a seguir
Likely follow-ups: "how would you monitor Kafka?" and "what would you do if consumer lag were growing continuously?".
Related chapters