The Dead Letter Queue Escape Hatch

What happens when a Kafka consumer can't process a message, and how it avoids getting stuck forever.

1 The happy path

message ✅ processed OK next message

Most messages arrive, get processed right away, and the consumer moves on. No drama.

2 But sometimes processing fails

message ❌ error raised retry it

A bug, a bad value, a missing record: whatever the cause, the consumer can't finish the job. It does not just give up on the first try.

3 Retry with a longer wait each time

1 retry wait 2s 2 retry wait 4s ❌ still failing limit reached

Each retry waits longer than the last (this is called backoff). Maybe the problem was temporary and just needed time to clear.

4 Out of retries: the DLQ escape hatch

❌ retries used up 🚪 moved to DLQ topic set aside, not lost next message

The troublesome message gets copied to a separate DLQ topic, parked for someone to look at later. The consumer is now free to keep going.

Without a DLQ, the consumer would keep retrying the same stuck message forever, and every message behind it would just have to wait.

5 The two paths, side by side

Happy path

message → processed → done → next message

Failure path

message → fails → retry, retry → still fails → DLQ topic → next message