grpc resource_error ai_generated true

INTERNAL: grpc: Channel connectivity watch failed: Too many pending watches

ID: grpc/channel-connectivity-watch-failure

Also available as: JSON · Markdown · 中文
85%Fix Rate
81%Confidence
1Evidence
2024-06-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC C++ 1.52.0 active — — —
gRPC Core 1.53.0 active — — —

Root Cause

Too many concurrent connectivity watches were registered on the same channel, exceeding the internal limit.

generic

中文

同一通道上注册了太多并发连接状态监视，超过了内部限制。

Official Documentation

https://grpc.io/docs/guides/performance/

Workarounds

  1. 85% success Limit the number of concurrent watches per channel. In C++, use `channel->GetState(true)` instead of multiple `NotifyOnStateChange` calls. 
    Limit the number of concurrent watches per channel. In C++, use `channel->GetState(true)` instead of multiple `NotifyOnStateChange` calls.
  2. 75% success Increase the limit via `GRPC_ARG_MAX_PENDING_WATCHES` (if available) or use a separate channel per watch group. 
    Increase the limit via `GRPC_ARG_MAX_PENDING_WATCHES` (if available) or use a separate channel per watch group.

中文步骤

  1. Limit the number of concurrent watches per channel. In C++, use `channel->GetState(true)` instead of multiple `NotifyOnStateChange` calls.
  2. Increase the limit via `GRPC_ARG_MAX_PENDING_WATCHES` (if available) or use a separate channel per watch group.

Dead Ends

Common approaches that don't work:

  1. 50% fail

    The limit is per channel; creating more channels may exhaust other resources.

  2. 70% fail

    This reduces the timeout but does not prevent the limit from being hit.

  3. 80% fail

    The watch failure may cause missed connectivity changes, leading to RPC failures.