GRPC_STREAM_IDLE_TIMEOUT grpc runtime_error ai_generated true

INTERNAL: grpc: stream 45 closed with error: stream idled out

ID: grpc/stream-idle-timeout-reset

Also available as: JSON · Markdown · 中文
82%Fix Rate
85%Confidence
1Evidence
2024-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC v1.50.x active — — —
gRPC v1.62.x active — — —
gRPC v1.64.x active — — —

Root Cause

gRPC server closed an idle stream due to per-stream idle timeout configuration, often caused by client not sending data within the timeout window.

generic

中文

gRPC 服务器因每流空闲超时配置关闭了空闲流，通常由客户端在超时窗口内未发送数据引起。

Official Documentation

https://grpc.io/docs/guides/keepalive/

Workarounds

  1. 85% success Increase the server-side stream idle timeout using gRPC server option: `server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_connection_idle_ms', 60000)])` in Python, or set `GRPC_ARG_MAX_CONNECTION_IDLE_MS` in C++. This prevents idle streams from being closed too aggressively. 
    Increase the server-side stream idle timeout using gRPC server option: `server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), options=[('grpc.max_connection_idle_ms', 60000)])` in Python, or set `GRPC_ARG_MAX_CONNECTION_IDLE_MS` in C++. This prevents idle streams from being closed too aggressively.
  2. 80% success Ensure the client sends periodic data or uses bidirectional streaming with a heartbeat message every 10-15 seconds to keep the stream active. 
    Ensure the client sends periodic data or uses bidirectional streaming with a heartbeat message every 10-15 seconds to keep the stream active.
  3. 75% success Reduce the server's `max_connection_age_ms` and `max_connection_age_grace_ms` to force connection rebalancing, which can reset idle counters. 
    Reduce the server's `max_connection_age_ms` and `max_connection_age_grace_ms` to force connection rebalancing, which can reset idle counters.

中文步骤

  1. 增加服务器端流空闲超时时间：在 Python 中设置 `grpc.max_connection_idle_ms` 为 60000 毫秒，或 C++ 中设置 `GRPC_ARG_MAX_CONNECTION_IDLE_MS`，避免空闲流过早关闭。
  2. 客户端定期发送数据或心跳消息（每 10-15 秒），保持流活跃。
  3. 降低服务器 `max_connection_age_ms` 和 `max_connection_age_grace_ms`，强制连接重新平衡，重置空闲计数器。

Dead Ends

Common approaches that don't work:

  1. 70% fail

    Keepalive pings prevent connection-level idle, but per-stream idle timeouts are separate and not affected by keepalive pings.

  2. 60% fail

    This controls minimum allowed ping interval, not stream idle timeout; it doesn't prevent stream closure due to inactivity.

  3. 50% fail

    Removing keepalive can cause connection drops, but stream idle timeout is a separate server-side policy that still applies.