UNAVAILABLE api network_error ai_generated partial

gRPC error: UNAVAILABLE. DNS resolution failed for service: my-service.example.com:443

ID: api/grpc-unavailable-dns-resolution

Also available as: JSON · Markdown · 中文
80%Fix Rate
87%Confidence
1Evidence
2024-01-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC v1.56.0 active — — —
Envoy proxy 1.27 active — — —
Kubernetes 1.28 active — — —

Root Cause

gRPC client cannot resolve the target service hostname due to DNS configuration issues or network connectivity problems.

generic

中文

gRPC客户端无法解析目标服务主机名，由于DNS配置问题或网络连接问题。

Official Documentation

https://grpc.github.io/grpc/core/md_doc_dns-resolver.html

Workarounds

  1. 85% success Verify DNS resolution manually: run 'dig my-service.example.com' or 'nslookup my-service.example.com'. If it fails, check /etc/resolv.conf for valid nameservers (e.g., 8.8.8.8). 
    Verify DNS resolution manually: run 'dig my-service.example.com' or 'nslookup my-service.example.com'. If it fails, check /etc/resolv.conf for valid nameservers (e.g., 8.8.8.8).
  2. 70% success Increase gRPC DNS resolver timeout: set GRPC_DNS_RESOLVER_TIMEOUT_MS=5000 environment variable before starting the client. 
    Increase gRPC DNS resolver timeout: set GRPC_DNS_RESOLVER_TIMEOUT_MS=5000 environment variable before starting the client.
  3. 80% success If using Kubernetes, ensure the service DNS name matches the gRPC target (e.g., my-service.namespace.svc.cluster.local). Use 'kubectl run -it --rm debug --image=busybox -- nslookup my-service' to test. 
    If using Kubernetes, ensure the service DNS name matches the gRPC target (e.g., my-service.namespace.svc.cluster.local). Use 'kubectl run -it --rm debug --image=busybox -- nslookup my-service' to test.

中文步骤

  1. Verify DNS resolution manually: run 'dig my-service.example.com' or 'nslookup my-service.example.com'. If it fails, check /etc/resolv.conf for valid nameservers (e.g., 8.8.8.8).
  2. Increase gRPC DNS resolver timeout: set GRPC_DNS_RESOLVER_TIMEOUT_MS=5000 environment variable before starting the client.
  3. If using Kubernetes, ensure the service DNS name matches the gRPC target (e.g., my-service.namespace.svc.cluster.local). Use 'kubectl run -it --rm debug --image=busybox -- nslookup my-service' to test.

Dead Ends

Common approaches that don't work:

  1. 75% fail

    Restarting the gRPC client without fixing DNS configuration (e.g., missing /etc/resolv.conf entries) will reproduce the same error.

  2. 60% fail

    Changing the target endpoint to an IP address bypasses DNS but may fail if the service uses virtual hosting or TLS SNI requires the hostname.