kafka runtime_error ai_generated true

org.apache.kafka.common.errors.PartitionNotFoundException: This server does not host this partition

ID: kafka/partition-not-found

Also available as: JSON · Markdown · 中文
78%Fix Rate
84%Confidence
1Evidence
2024-05-12First Seen

Root Cause

Client request targets a partition that is not hosted on the broker, often due to stale metadata or partition reassignment.

generic

中文

客户端请求的目标分区不在该代理上托管，通常由于元数据过时或分区重新分配。

Official Documentation

https://kafka.apache.org/documentation/#partition_reassignment

Workarounds

  1. 75% success Force metadata refresh by setting metadata.max.age.ms to a lower value, e.g., props.put("metadata.max.age.ms", "5000"); 
    Force metadata refresh by setting metadata.max.age.ms to a lower value, e.g., props.put("metadata.max.age.ms", "5000");
  2. 85% success Wait for partition reassignment to complete and check status: kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --verify --reassignment-json-file reassign.json 
    Wait for partition reassignment to complete and check status: kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --verify --reassignment-json-file reassign.json
  3. 70% success Restart the client to force a fresh metadata fetch from the broker. 
    Restart the client to force a fresh metadata fetch from the broker.

中文步骤

  1. 通过设置 metadata.max.age.ms 为较低值强制刷新元数据，例如 props.put("metadata.max.age.ms", "5000");
  2. 等待分区重新分配完成并检查状态：kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --verify --reassignment-json-file reassign.json
  3. 重启客户端以强制从代理获取新的元数据。

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Manually deleting the partition metadata can lead to data loss and inconsistency.

  2. 90% fail

    Ignoring the error and retrying without refreshing metadata will keep failing.