kafka runtime_error ai_generated true

org.apache.kafka.common.errors.OffsetCommitRejectedException: Offset commit rejected due to fenced epoch

ID: kafka/offset-commit-rejected-fenced

Also available as: JSON · Markdown · 中文
82%Fix Rate
86%Confidence
1Evidence
2024-01-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.5.0 active — — —
3.6.0 active — — —

Root Cause

Consumer group coordinator rejected offset commit because the consumer's epoch is stale due to a newer consumer instance taking over.

generic

中文

消费者组协调器拒绝了偏移量提交，因为消费者的时代已过时，这是由于较新的消费者实例接管导致的。

Official Documentation

https://kafka.apache.org/documentation/#consumerconfigs_group.id

Workarounds

  1. 90% success Ensure unique group.instance.id per consumer instance to avoid epoch conflicts; use UUID-based IDs. 
    Ensure unique group.instance.id per consumer instance to avoid epoch conflicts; use UUID-based IDs.
  2. 85% success Catch OffsetCommitRejectedException and rejoin the group with a fresh consumer instance. 
    Catch OffsetCommitRejectedException and rejoin the group with a fresh consumer instance.
  3. 80% success Set 'group.protocol' to 'consumer' and disable static membership by removing group.instance.id. 
    Set 'group.protocol' to 'consumer' and disable static membership by removing group.instance.id.

中文步骤

  1. 确保每个消费者实例使用唯一的 group.instance.id，以避免时代冲突；使用基于UUID的ID。
  2. 捕获 OffsetCommitRejectedException 并使用新的消费者实例重新加入组。
  3. 设置 'group.protocol' 为 'consumer'，并通过移除 group.instance.id 禁用静态成员。

Dead Ends

Common approaches that don't work:

  1. 85% fail

    Manual commit still uses same epoch; coordinator rejects if epoch is fenced.

  2. 70% fail

    Longer interval delays rebalance but does not fix epoch fencing from duplicate consumers.

  3. 60% fail

    Deleting offsets resets group but does not prevent future fencing if same consumer ID is used.