kafka runtime_error ai_generated true

org.apache.kafka.common.errors.DeadLetterQueueException: Failed to send record to dead letter topic DLQ_topic

ID: kafka/dead-letter-queue-producer-exception-on-send

Also available as: JSON · Markdown · 中文
76%Fix Rate
84%Confidence
1Evidence
2024-02-28First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Kafka 3.4.0 active — — —
Kafka 3.6.0 active — — —
Kafka 3.7.0 active — — —

Root Cause

A Kafka Streams or Kafka Connect dead letter queue producer encountered an error when attempting to forward a failed record to the configured DLQ topic, often due to serialization or ACL issues.

generic

中文

Kafka Streams或Kafka Connect死信队列生产者在尝试将失败记录转发到配置的DLQ主题时遇到错误，通常是由于序列化或ACL问题。

Official Documentation

https://kafka.apache.org/documentation/streams/developer-guide/config-streams.html#dead-letter-queue

Workarounds

  1. 88% success Verify the DLQ topic exists and has correct ACLs using 'bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic DLQ_topic' and 'bin/kafka-acls.sh --bootstrap-server localhost:9092 --list --topic DLQ_topic'. 
    Verify the DLQ topic exists and has correct ACLs using 'bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic DLQ_topic' and 'bin/kafka-acls.sh --bootstrap-server localhost:9092 --list --topic DLQ_topic'.
  2. 82% success Ensure the DLQ producer uses the same key/value serializers as the main topic. In Kafka Streams, set 'dlq.topic.name' and configure 'default.key.serde' and 'default.value.serde' correctly. 
    Ensure the DLQ producer uses the same key/value serializers as the main topic. In Kafka Streams, set 'dlq.topic.name' and configure 'default.key.serde' and 'default.value.serde' correctly.

中文步骤

  1. Verify the DLQ topic exists and has correct ACLs using 'bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic DLQ_topic' and 'bin/kafka-acls.sh --bootstrap-server localhost:9092 --list --topic DLQ_topic'.
  2. Ensure the DLQ producer uses the same key/value serializers as the main topic. In Kafka Streams, set 'dlq.topic.name' and configure 'default.key.serde' and 'default.value.serde' correctly.

Dead Ends

Common approaches that don't work:

  1. Setting producer retries to Integer.MAX_VALUE 85% fail

    Increasing producer retries only delays the eventual failure; the root cause is usually misconfiguration.

  2. Removing the DLQ configuration from the connector 95% fail

    Disabling the DLQ entirely loses the failed records and defeats the purpose of error handling.