42 Kafka interview questions with answers

All Kafka interview questions

• What is Apache Kafka and why is it widely used in distributed systems?

• Explain Kafka's architecture: how do Broker, Topic, Partition, and Consumer Group work?

• How do Producers and Consumers work in Kafka? What are the key configurations to know?

• What is offset management in Kafka? What is the difference between auto commit and manual commit?

• How is message ordering guaranteed in Kafka? When can ordering be broken?

• What are Replication and ISR (In-Sync Replicas) in Kafka? How does Kafka ensure fault tolerance?

• What is Kafka Connect? What problem does it solve in data pipelines?

• What is Kafka Streams? How does it differ from regular message consumption?

• What are the most common real-world use cases for Kafka?

• How does exactly-once semantics (EOS) work in Kafka? Explain idempotent producers and transactions.

• Partitioning strategies in Kafka: when should you use key-based, round-robin, or a custom partitioner?

• Consumer rebalancing in Kafka: what issues can occur and how do you optimize it?

• What is a Dead Letter Queue (DLQ) in Kafka? How do you implement error message handling?

• Kafka monitoring and performance tuning: what are the key metrics and how do you optimize throughput/latency?

• Kafka in microservices architecture: event-driven design, the saga pattern, and pitfalls to avoid?

• What is Schema Registry in the Kafka ecosystem? Why use Avro/Protobuf instead of JSON?

• What is KRaft mode in Kafka? Why is Kafka removing ZooKeeper?

• What is an idempotent producer in Kafka? How do you enable it and how does it work?

• How does Kafka producer batching work? How do linger.ms and batch.size optimize throughput?

• At-least-once vs at-most-once vs exactly-once in Kafka: differences and how to achieve each?

• Consumer groups and partition assignment: how does Kafka distribute partitions to consumers?

• What is Kafka consumer lag? How do you monitor and alert when lag is too high?

• What is log compaction in Kafka? What are tombstone records used for?

• How do min.insync.replicas and acks=all work together to protect data? When does a producer throw NotEnoughReplicasException?

• What is MirrorMaker 2? How do you perform cross-cluster replication in Kafka?

• What is ksqlDB? How does it differ from Kafka Streams?

• Event sourcing with Kafka: how does the pattern work and what are the benefits and drawbacks?

• What is the Transactional Outbox Pattern? Why is it necessary when using Kafka with a database?

• Kafka vs RabbitMQ: detailed trade-off comparison to choose the right one for each use case?

• CQRS pattern with Kafka: how are Command side and Query side separated?

• Partition key design in Kafka: how do you choose keys to avoid hotspots and ensure ordering?

• What is a message queue and when should you use one?

• How does Kafka handle data retention?

• How does the Kafka producer compress data, and what are the trade-offs?

• Kafka Streams vs Apache Flink — when do you pick which?

• What is the difference between at-most-once, at-least-once, and exactly-once? Which one should you pick in practice?

• What is a DLQ for? What is the production process for handling messages that land in it?

• How do you guarantee message ordering? How do you pick a partition key and what are the trade-offs?

• What is consumer lag and how do you measure it? What do you do when lag keeps growing?

• What is a consumer group rebalance? What impact does it have and how do you reduce it?

• How does Kafka provide exactly-once, and where are its limits?

• How do you evolve a message schema without breaking consumers? What is the difference between backward and forward compatibility?