kafka system_error ai_generated partial

AdminCommandFailedException: Partition reassignment failed due to no viable candidates

ID: kafka/partition-reassignment-failed-no-candidates

Also available as: JSON · Markdown · 中文
70%Fix Rate
83%Confidence
1Evidence
2024-01-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Kafka 3.2.0 active — — —
Kafka 3.5.0 active — — —
Kafka 3.7.0 active — — —

Root Cause

The partition reassignment algorithm cannot find any broker that satisfies the specified constraints (e.g., rack awareness, resource limits).

generic

中文

分区重新分配算法无法找到任何满足指定约束条件（如机架感知、资源限制）的代理。

Official Documentation

https://kafka.apache.org/documentation/#basic_ops_cluster_expansion

Workarounds

  1. 75% success Add more brokers to the cluster or free up resources on existing brokers (e.g., delete unused topics, increase disk space). Then retry the reassignment with a balanced plan. 
    Add more brokers to the cluster or free up resources on existing brokers (e.g., delete unused topics, increase disk space). Then retry the reassignment with a balanced plan.
  2. 80% success Use the `--generate` option in `kafka-reassign-partitions.sh` to create a reassignment plan that respects current constraints, then execute it with `--execute`. 
    Use the `--generate` option in `kafka-reassign-partitions.sh` to create a reassignment plan that respects current constraints, then execute it with `--execute`.

中文步骤

  1. Add more brokers to the cluster or free up resources on existing brokers (e.g., delete unused topics, increase disk space). Then retry the reassignment with a balanced plan.
  2. Use the `--generate` option in `kafka-reassign-partitions.sh` to create a reassignment plan that respects current constraints, then execute it with `--execute`.

Dead Ends

Common approaches that don't work:

  1. Forcing reassignment with `--force` flag without adjusting constraints 90% fail

    The `--force` flag does not bypass viability checks; it only skips some safety checks, and the algorithm still fails.

  2. Manually specifying a broker that is already at full disk capacity 85% fail

    The broker cannot accept more partitions due to resource limits, and the reassignment will fail or degrade cluster performance.