kafka runtime_error ai_generated true

org.apache.kafka.common.errors.DeadLetterQueueException：无法发送记录到死信主题DLQ_topic

org.apache.kafka.common.errors.DeadLetterQueueException: Failed to send record to dead letter topic DLQ_topic

ID: kafka/dead-letter-queue-producer-exception-on-send

其他格式: JSON · Markdown 中文 · English
76%修复率
84%置信度
1证据数
2024-02-28首次发现

版本兼容性

版本状态引入弃用备注
Kafka 3.4.0 active — — —
Kafka 3.6.0 active — — —
Kafka 3.7.0 active — — —

根因分析

Kafka Streams或Kafka Connect死信队列生产者在尝试将失败记录转发到配置的DLQ主题时遇到错误，通常是由于序列化或ACL问题。

English

A Kafka Streams or Kafka Connect dead letter queue producer encountered an error when attempting to forward a failed record to the configured DLQ topic, often due to serialization or ACL issues.

generic

官方文档

https://kafka.apache.org/documentation/streams/developer-guide/config-streams.html#dead-letter-queue

解决方案

  1. Verify the DLQ topic exists and has correct ACLs using 'bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic DLQ_topic' and 'bin/kafka-acls.sh --bootstrap-server localhost:9092 --list --topic DLQ_topic'.
  2. Ensure the DLQ producer uses the same key/value serializers as the main topic. In Kafka Streams, set 'dlq.topic.name' and configure 'default.key.serde' and 'default.value.serde' correctly.

无效尝试

常见但无效的做法:

  1. Setting producer retries to Integer.MAX_VALUE 85% 失败

    Increasing producer retries only delays the eventual failure; the root cause is usually misconfiguration.

  2. Removing the DLQ configuration from the connector 95% 失败

    Disabling the DLQ entirely loses the failed records and defeats the purpose of error handling.