go network_error ai_generated true

rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: error while dialing: dial tcp 10.0.0.1:8080: i/o timeout"

ID: go/grpc-unavailable-load-balancer-fail

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
0Evidence
2024-09-14First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
1.0 active — — —
1.1 active — — —

Root Cause

All backend servers behind a load balancer are unreachable, often due to network partition or server crash.

generic

中文

负载均衡器后面的所有后端服务器都不可达，通常是由于网络分区或服务器崩溃。

Workarounds

  1. 90% success Check the health of all backend servers and ensure they are reachable. 
    Use curl or telnet to test connectivity: telnet 10.0.0.1 8080, and check server logs.
  2. 80% success Implement a fallback to a different load balancer or DNS resolver. 
    resolver := resolver.NewBuilder("dns:///example.com:8080")
conn, err := grpc.Dial("dns:///example.com:8080", grpc.WithInsecure(), grpc.WithResolvers(resolver))

Dead Ends

Common approaches that don't work:

  1. Restart the client application. 95% fail

    The client is fine; the issue is with the servers or network.

  2. Increase the dial timeout to a very high value. 80% fail

    If servers are down, longer timeout just delays the error.