kafka network_error ai_generated partial

org.apache.kafka.common.errors.FetchTimeoutException：分区my-topic-0的获取请求在30000毫秒后超时

org.apache.kafka.common.errors.FetchTimeoutException: Fetch request timed out after 30000 ms for partition my-topic-0

ID: kafka/consumer-fetch-timeout

其他格式: JSON · Markdown 中文 · English
75%修复率
82%置信度
1证据数
2023-11-20首次发现

版本兼容性

版本状态引入弃用备注
kafka 3.0.0 active — — —
kafka 3.3.0 active — — —
kafka 3.5.0 active — — —

根因分析

由于网络拥塞、代理过载或分区领导者缓慢，消费者对代理的获取请求超过了配置的超时时间。

English

The consumer's fetch request to the broker exceeded the configured timeout due to network congestion, broker overload, or a slow partition leader.

generic

官方文档

https://kafka.apache.org/documentation/#consumerconfigs_fetch.max.wait.ms

解决方案

  1. Increase the broker's 'fetch.purgatory.purge.interval.requests' and tune 'num.network.threads' to handle more concurrent fetch requests, reducing queue delays.
  2. Add a retry mechanism in the consumer with exponential backoff on FetchTimeoutException, and reduce fetch.min.bytes to allow smaller, faster responses.

无效尝试

常见但无效的做法:

  1. 40% 失败

    This only masks the timeout and can lead to increased consumer latency and backlog, without fixing the underlying network or broker issue.

  2. 70% 失败

    Restarting re-establishes the connection but does not resolve persistent network or broker load issues; the timeout will recur.