grpc network_error ai_generated partial

UNAVAILABLE: 负载均衡器移除了流

UNAVAILABLE: stream removed by load balancer

ID: grpc/stream-removed-by-load-balancer

其他格式: JSON · Markdown 中文 · English
78%修复率
85%置信度
1证据数
2024-06-15首次发现

版本兼容性

版本状态引入弃用备注
gRPC v1.62.0 active — — —
Envoy v1.29.0 active — — —
gRPC-LB v1.8.0 active — — —

根因分析

gRPC负载均衡器（如gRPC-LB或Envoy）因后端健康检查失败或连接排空而移除了活跃流。

English

gRPC load balancer (e.g., gRPC-LB or Envoy) removed an active stream due to backend health check failure or connection draining.

generic

官方文档

https://grpc.io/docs/guides/load-balancing/

解决方案

  1. 配置gRPC客户端重试策略和回退：设置环境变量'grpc.enable_retries=1'和'GRPC_RETRY_BUFFER_SIZE=32MB'；实现自定义重试拦截器，使用指数退避（如100ms、500ms、2s）。
  2. 调整负载均衡器健康检查间隔和超时：在Envoy中设置'health_check.interval: 5s'和'health_check.timeout: 2s'以减少误移除；对关键流设置'connection_drain_on_health_failure: false'。
  3. 实现客户端重连逻辑：使用gRPC的'Channelz'监控子通道状态，在'TRANSIENT_FAILURE'时重连（如'channel.watch_connectivity_state(grpc.ChannelConnectivity.TRANSIENT_FAILURE)'）。

无效尝试

常见但无效的做法:

  1. 85% 失败

    Increasing client retry count without addressing load balancer health check fails because retries hit the same unhealthy backend.

  2. 70% 失败

    Disabling TLS entirely reduces security and may mask underlying connection issues; the error persists if the load balancer still sees backend as unhealthy.

  3. 90% 失败

    Restarting the client without adjusting load balancer configuration only temporarily reconnects; the error recurs on next health check failure.