GRPC_MAX_CONCURRENT_STREAMS grpc resource_error ai_generated partial

INTERNAL: grpc: max concurrent streams exceeded on server: 128 active streams, limit 100

ID: grpc/max-concurrent-streams-exceeded

Also available as: JSON · Markdown · 中文
82%Fix Rate
84%Confidence
1Evidence
2023-09-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC-go v1.58.0 active — — —
gRPC-java v1.57.0 active — — —
C-core v1.58.0 active — — —

Root Cause

The gRPC server has reached its configured maximum number of concurrent streams (default 100 in many implementations) and rejected new stream creation due to resource exhaustion.

generic

中文

gRPC 服务器已达到配置的最大并发流数（许多实现中默认 100），由于资源耗尽而拒绝创建新流。

Official Documentation

https://grpc.io/docs/guides/performance/#streams

Workarounds

  1. 85% success Increase the max concurrent streams limit in server configuration. In Go: server := grpc.NewServer(grpc.MaxConcurrentStreams(200)). In Java: NettyServerBuilder.forPort(port).maxConcurrentCallsPerConnection(200). 
    Increase the max concurrent streams limit in server configuration. In Go: server := grpc.NewServer(grpc.MaxConcurrentStreams(200)). In Java: NettyServerBuilder.forPort(port).maxConcurrentCallsPerConnection(200).
  2. 80% success Enable client-side connection multiplexing by setting grpc.initial_conn_window_size to a higher value (e.g., 1MB) and use a load balancer to distribute streams across multiple server connections. 
    Enable client-side connection multiplexing by setting grpc.initial_conn_window_size to a higher value (e.g., 1MB) and use a load balancer to distribute streams across multiple server connections.
  3. 75% success Implement stream pooling or batching on the client: instead of opening one stream per request, use client-side streaming to send multiple messages over a single stream, reducing the total stream count. 
    Implement stream pooling or batching on the client: instead of opening one stream per request, use client-side streaming to send multiple messages over a single stream, reducing the total stream count.

中文步骤

  1. Increase the max concurrent streams limit in server configuration. In Go: server := grpc.NewServer(grpc.MaxConcurrentStreams(200)). In Java: NettyServerBuilder.forPort(port).maxConcurrentCallsPerConnection(200).
  2. Enable client-side connection multiplexing by setting grpc.initial_conn_window_size to a higher value (e.g., 1MB) and use a load balancer to distribute streams across multiple server connections.
  3. Implement stream pooling or batching on the client: instead of opening one stream per request, use client-side streaming to send multiple messages over a single stream, reducing the total stream count.

Dead Ends

Common approaches that don't work:

  1. 80% fail

    Simply increasing the limit without addressing slow consumers or long-lived streams can lead to memory exhaustion and OOM kills.

  2. 75% fail

    Adding more server instances without adjusting client connection pooling may cause clients to still hit per-connection limits on each instance.