FailedCreatePodSandBox cloud resource_error ai_generated true

Warning FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "" network for pod "my-pod": networkPlugin cni failed to set up pod "my-pod_default" network: no IP addresses available in network

ID: cloud/azure-aks-pod-creation-pending-no-ip

Also available as: JSON · Markdown · 中文
85%Fix Rate
87%Confidence
1Evidence
2023-09-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
AKS 1.26 active — — —
AKS 1.27 active — — —
Azure CNI active — — —

Root Cause

Azure Kubernetes Service (AKS) node pool has exhausted its subnet IP address range, preventing new pods from being assigned IP addresses.

generic

中文

Azure Kubernetes Service (AKS)节点池已耗尽子网IP地址范围，阻止新Pod分配IP地址。

Official Documentation

https://learn.microsoft.com/en-us/azure/aks/use-multiple-node-pools

Workarounds

  1. 90% success Increase the subnet size by adding a new subnet and updating the AKS cluster to use it. First, create a new subnet: `az network vnet subnet create --resource-group my-rg --vnet-name my-vnet --name new-subnet --address-prefixes 10.0.1.0/24` Then update the node pool: `az aks nodepool update --resource-group my-rg --cluster-name my-cluster --name my-nodepool --vnet-subnet-id /subscriptions/.../subnets/new-subnet` 
    Increase the subnet size by adding a new subnet and updating the AKS cluster to use it. First, create a new subnet: `az network vnet subnet create --resource-group my-rg --vnet-name my-vnet --name new-subnet --address-prefixes 10.0.1.0/24` Then update the node pool: `az aks nodepool update --resource-group my-rg --cluster-name my-cluster --name my-nodepool --vnet-subnet-id /subscriptions/.../subnets/new-subnet`
  2. 70% success Delete unused pods to free IPs. Use `kubectl delete pod <pod-name>` for non-critical pods. Then monitor with `kubectl get pods -o wide` to see IP reclamation. 
    Delete unused pods to free IPs. Use `kubectl delete pod <pod-name>` for non-critical pods. Then monitor with `kubectl get pods -o wide` to see IP reclamation.

中文步骤

  1. Increase the subnet size by adding a new subnet and updating the AKS cluster to use it. First, create a new subnet: `az network vnet subnet create --resource-group my-rg --vnet-name my-vnet --name new-subnet --address-prefixes 10.0.1.0/24` Then update the node pool: `az aks nodepool update --resource-group my-rg --cluster-name my-cluster --name my-nodepool --vnet-subnet-id /subscriptions/.../subnets/new-subnet`
  2. Delete unused pods to free IPs. Use `kubectl delete pod <pod-name>` for non-critical pods. Then monitor with `kubectl get pods -o wide` to see IP reclamation.

Dead Ends

Common approaches that don't work:

  1. Restart the AKS cluster 95% fail

    Restarting does not free up IP addresses; the subnet is still full.

  2. Scale down the node pool to reduce node count 80% fail

    Scaling down reduces nodes but does not reclaim IPs from existing pods. Pod IPs remain allocated until pods are deleted.