GRPC_MAX_CONCURRENT_STREAMS grpc resource_error ai_generated true

UNAVAILABLE: grpc: server rejected stream due to max concurrent streams limit (100)

ID: grpc/max-concurrent-streams-server

Also available as: JSON · Markdown · 中文
88%Fix Rate
87%Confidence
1Evidence
2024-06-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC v1.52.x active — — —
gRPC v1.60.x active — — —
gRPC v1.65.x active — — —

Root Cause

Server has reached its configured maximum number of concurrent streams (e.g., 100), and new stream requests are rejected until existing streams complete.

generic

中文

服务器已达到配置的最大并发流数（例如 100），新流请求被拒绝，直到现有流完成。

Official Documentation

https://grpc.io/docs/guides/performance/#server-streams

Workarounds

  1. 90% success Increase the server's max concurrent streams: set `GRPC_ARG_HTTP2_MAX_STREAMS` to a higher value (e.g., 500) in the server builder. In C++: `channel_args->SetInt(GRPC_ARG_HTTP2_MAX_STREAMS, 500);` In Python: `grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_concurrent_streams', 500)])`. 
    Increase the server's max concurrent streams: set `GRPC_ARG_HTTP2_MAX_STREAMS` to a higher value (e.g., 500) in the server builder. In C++: `channel_args->SetInt(GRPC_ARG_HTTP2_MAX_STREAMS, 500);` In Python: `grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_concurrent_streams', 500)])`.
  2. 85% success Implement a client-side connection pool: create multiple gRPC channels (e.g., 3-5) and distribute streams across them to avoid hitting the limit on a single channel. 
    Implement a client-side connection pool: create multiple gRPC channels (e.g., 3-5) and distribute streams across them to avoid hitting the limit on a single channel.
  3. 80% success Use a load balancer (e.g., Envoy) to distribute streams across multiple server instances, effectively increasing the total stream capacity. 
    Use a load balancer (e.g., Envoy) to distribute streams across multiple server instances, effectively increasing the total stream capacity.

中文步骤

  1. 增加服务器最大并发流数：设置 `GRPC_ARG_HTTP2_MAX_STREAMS` 为更高值（如 500）。在 Python 中：`grpc.server(..., options=[('grpc.max_concurrent_streams', 500)])`。
  2. 实现客户端连接池：创建多个 gRPC 通道（如 3-5 个），将流分布到不同通道，避免单个通道达到限制。
  3. 使用负载均衡器（如 Envoy）将流分发到多个服务器实例，有效增加总流容量。

Dead Ends

Common approaches that don't work:

  1. 75% fail

    Retries will still be rejected as long as the server is at capacity; it only adds latency without solving the bottleneck.

  2. 60% fail

    Keepalive pings do not affect stream limits; they only maintain connection health.

  3. 70% fail

    This setting controls the client's advertised limit, not the server's; the server independently enforces its own limit.