grpc resource_error ai_generated true

INTERNAL: grpc: stream creation failed due to max concurrent streams limit (128) on server

ID: grpc/stream-creation-failure-max-streams

Also available as: JSON · Markdown · 中文
75%Fix Rate
84%Confidence
1Evidence
2024-04-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC C++ 1.63.0 active — — —
gRPC Java 1.61.0 active — — —
gRPC Python 1.64.0 active — — —

Root Cause

The gRPC server reached its configured maximum number of concurrent streams (default 100 or 128), rejecting new stream creation.

generic

中文

gRPC 服务器达到了配置的最大并发流数（默认 100 或 128），拒绝创建新流。

Official Documentation

https://grpc.io/docs/guides/performance/#http2-settings

Workarounds

  1. 80% success Increase `GRPC_ARG_HTTP2_MAX_CONCURRENT_STREAMS` to a higher value (e.g., 256) on server: `server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_concurrent_streams', 256)])` 
    Increase `GRPC_ARG_HTTP2_MAX_CONCURRENT_STREAMS` to a higher value (e.g., 256) on server: `server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_concurrent_streams', 256)])`
  2. 75% success Implement client-side connection pooling to reuse streams: `channel = grpc.insecure_channel('localhost:50051', options=[('grpc.use_local_subchannel_pool', 1)])` 
    Implement client-side connection pooling to reuse streams: `channel = grpc.insecure_channel('localhost:50051', options=[('grpc.use_local_subchannel_pool', 1)])`

中文步骤

  1. 在服务器端增加 `GRPC_ARG_HTTP2_MAX_CONCURRENT_STREAMS` 到更高值（如 256）：`server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_concurrent_streams', 256)])`
  2. 实现客户端连接池以复用流：`channel = grpc.insecure_channel('localhost:50051', options=[('grpc.use_local_subchannel_pool', 1)])`

Dead Ends

Common approaches that don't work:

  1. Increase max streams limit to an extremely high value like 10000 60% fail

    May cause memory exhaustion and degrade performance; doesn't address inefficient stream usage.

  2. Disable the stream limit entirely by setting to 0 70% fail

    gRPC interprets 0 as unlimited, but may cause resource starvation and DoS vulnerability.

  3. Restart server every time it hits limit 90% fail

    Temporary fix; the issue recurs as traffic grows.