UNAVAILABLE api network_error ai_generated partial

gRPC error: UNAVAILABLE: DNS resolution failed for service

ID: api/grpc-unavailable-dns-resolution-failed

Also available as: JSON · Markdown · 中文
78%Fix Rate
82%Confidence
1Evidence
2024-01-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC v1.58.0 active — — —
Envoy 1.28.0 active — — —
Kubernetes 1.28 active — — —
CoreDNS 1.11.0 active — — —

Root Cause

gRPC client cannot resolve the service hostname via DNS, often due to transient network issues, misconfigured DNS, or service discovery outages.

generic

中文

gRPC客户端无法通过DNS解析服务主机名，通常是由于临时网络问题、DNS配置错误或服务发现中断。

Official Documentation

https://grpc.io/docs/guides/error/

Workarounds

  1. 90% success Verify DNS resolution: `nslookup service.example.com` or `dig service.example.com`. If fails, check /etc/resolv.conf and DNS server health. 
    Verify DNS resolution: `nslookup service.example.com` or `dig service.example.com`. If fails, check /etc/resolv.conf and DNS server health.
  2. 80% success Configure gRPC with custom DNS resolver: In Go, use `google.golang.org/grpc/resolver` and set `grpc.WithDefaultServiceConfig('{"loadBalancingConfig": [{"round_robin":{}}]}')` 
    Configure gRPC with custom DNS resolver: In Go, use `google.golang.org/grpc/resolver` and set `grpc.WithDefaultServiceConfig('{"loadBalancingConfig": [{"round_robin":{}}]}')`
  3. 85% success Add retry logic with exponential backoff for transient DNS failures: in gRPC-Go, use `grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor())` 
    Add retry logic with exponential backoff for transient DNS failures: in gRPC-Go, use `grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor())`

中文步骤

  1. 验证DNS解析：`nslookup service.example.com` 或 `dig service.example.com`。如果失败，检查/etc/resolv.conf和DNS服务器健康状态。
  2. 为gRPC配置自定义DNS解析器：在Go中，使用`google.golang.org/grpc/resolver`并设置`grpc.WithDefaultServiceConfig('{"loadBalancingConfig": [{"round_robin":{}}]}')`
  3. 添加带指数退避的重试逻辑以处理临时DNS故障：在gRPC-Go中，使用`grpc.WithUnaryInterceptor(grpc_retry.UnaryClientInterceptor())`

Dead Ends

Common approaches that don't work:

  1. 85% fail

    Restarting the client application without fixing DNS configuration or network connectivity

  2. 70% fail

    Assuming the service is down when it's actually a DNS resolution failure on the client side

  3. 60% fail

    Changing gRPC load balancing policy (e.g., round_robin to pick_first) which doesn't affect DNS resolution