kafka protocol_error ai_generated true

org.apache.kafka.common.errors.ProducerFencedException：由于分区重新分配，生产者纪元被隔离

org.apache.kafka.common.errors.ProducerFencedException: Producer epoch fenced due to partition reassignment

ID: kafka/transactional-producer-epoch-fenced-during-partition-reassignment

其他格式: JSON · Markdown 中文 · English
81%修复率
88%置信度
1证据数
2024-06-05首次发现

版本兼容性

版本状态引入弃用备注
Kafka 3.3.0 active — — —
Kafka 3.5.0 active — — —
Kafka 3.7.0 active — — —

根因分析

事务性生产者的纪元被无效化，因为它正在写入的分区被重新分配到不同的代理，导致事务协调器隔离了该生产者。

English

A transactional producer's epoch was invalidated because a partition it was writing to was reassigned to a different broker, causing the transaction coordinator to fence the producer.

generic

官方文档

https://kafka.apache.org/documentation/#producerconfigs_transactional.id

解决方案

  1. Initialize a new transactional producer with a new transactional.id after catching the exception. Example in Java: try { producer.initTransactions(); } catch (ProducerFencedException e) { producer.close(); producer = createNewProducerWithNewTransactionalId(); producer.initTransactions(); }
  2. Ensure partition reassignment is completed before starting transactional writes. Use 'bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --verify --reassignment-json-file reassign.json' to confirm completion.

无效尝试

常见但无效的做法:

  1. Setting transaction.timeout.ms to 300000 (5 minutes) 90% 失败

    Increasing transaction.timeout.ms does not prevent epoch fencing; it only delays transaction expiration.

  2. Restarting the producer application without changing transactional.id 95% 失败

    Restarting the producer without resetting the transactional.id will still use the fenced epoch.