ECS.ServiceConnect.DNSResolution cloud network_error ai_generated true

ECS Service Connect: DNS resolution failed for service endpoint within the same namespace

ID: cloud/aws-ecs-service-connect-dns-resolution-failure

Also available as: JSON · Markdown · 中文
78%Fix Rate
83%Confidence
1Evidence
2024-06-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
AWS ECS: latest active — — —
AWS Cloud Map: latest active — — —
Amazon VPC: any active — — —

Root Cause

When using ECS Service Connect, tasks fail to resolve the DNS name of a service in the same namespace because the service discovery namespace is not properly associated with the VPC, or the service's DNS records are not propagated due to a misconfigured service connect configuration.

generic

中文

使用 ECS Service Connect 时，任务无法解析同一命名空间中服务的 DNS 名称，因为服务发现命名空间未正确关联到 VPC，或者由于服务连接配置错误导致服务的 DNS 记录未传播。

Official Documentation

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect.html

Workarounds

  1. 85% success Verify that the Cloud Map namespace is associated with the correct VPC. Run 'aws servicediscovery list-namespaces' and check the VPC configuration. If missing, create a new namespace with 'aws servicediscovery create-http-namespace --name my-namespace --vpc vpc-12345'. 
    Verify that the Cloud Map namespace is associated with the correct VPC. Run 'aws servicediscovery list-namespaces' and check the VPC configuration. If missing, create a new namespace with 'aws servicediscovery create-http-namespace --name my-namespace --vpc vpc-12345'.
  2. 80% success Ensure the ECS task definition includes the 'awslogs' log driver and check the Service Connect logs for DNS queries using 'aws logs describe-log-groups' and filtering for 'DNS_RESOLUTION_FAILED'. Then update the service connect configuration with the correct 'clientAlias' and 'discoveryName'. 
    Ensure the ECS task definition includes the 'awslogs' log driver and check the Service Connect logs for DNS queries using 'aws logs describe-log-groups' and filtering for 'DNS_RESOLUTION_FAILED'. Then update the service connect configuration with the correct 'clientAlias' and 'discoveryName'.

中文步骤

  1. Verify that the Cloud Map namespace is associated with the correct VPC. Run 'aws servicediscovery list-namespaces' and check the VPC configuration. If missing, create a new namespace with 'aws servicediscovery create-http-namespace --name my-namespace --vpc vpc-12345'.
  2. Ensure the ECS task definition includes the 'awslogs' log driver and check the Service Connect logs for DNS queries using 'aws logs describe-log-groups' and filtering for 'DNS_RESOLUTION_FAILED'. Then update the service connect configuration with the correct 'clientAlias' and 'discoveryName'.

Dead Ends

Common approaches that don't work:

  1. 80% fail

    Overrides the default AWS DNS resolver, breaking Service Connect's built-in DNS resolution for the namespace.

  2. 70% fail

    IPs can change when tasks are redeployed or scaled; defeats the purpose of service discovery.

  3. 50% fail

    Restarting doesn't fix the underlying DNS configuration issue; the new tasks will face the same resolution failure.