Question 48 of 50
How would you monitor Kafka?
Question
"How would you monitor a Kafka pipeline in production?"
What the interviewer wants to assess
The typical last technical question in a Kafka interview — tests whether the candidate thinks about real operations, connecting metrics to decisions, not just naming tools.
Resposta rápida
I'd monitor consumer lag and throughput (to know whether consumption keeps up with production), DLQ message rate and error rate (to detect persistent failures), rebalance rate (to detect consumer instability), and structured logs with correlationId for cross-service traceability.
Resposta nível Sênior
No single metric tells the whole story. Growing consumer lag anticipates processing delay before the user
feels it; low throughput, combined with growing lag, points to consumption capacity being the bottleneck,
not abnormal production volume. DLQ message rate signals persistent failures that already got past retry —
a quality symptom, not a capacity one. Frequent rebalance rate usually indicates a misconfigured
max.poll.interval.ms or infrastructure instability, not Kafka itself. And structured logs with
correlationId, aggregated in a tool like Kibana or Datadog, let you reconstruct a specific transaction's
path across multiple services when an alert fires. The tool choice (open-source Prometheus+Grafana vs.
commercial Datadog/New Relic) is an infrastructure decision independent of this set of metrics.
In-depth explanation
See the full chapter — Observability.
Exemplo financeiro
The payments team monitors the saldo-service's consumer lag, the DLQ message rate on the pix.recebido
topic, and propagates correlationId from the original HTTP call to every consumer — letting them find,
within seconds during an incident, every service that processed a specific PIX.
"Monitoring Kafka is just installing Prometheus and Grafana"
Installing the tools is the easy step. What separates a senior engineer is knowing which metrics to monitor, which alert thresholds make sense for each business flow, and how to connect those metrics to real operational decisions.
Pode vir a seguir
Likely follow-ups: "walk me through a real project using Kafka" and "what errors or challenges have you faced with Kafka".
Related chapters