kubernetes scheduling_error ai_generated true

Warning FailedScheduling: 0/N nodes are available: N node(s) had untolerated taint {key: value}

ID: kubernetes/taint-toleration-not-matched

Also available as: JSON · Markdown
92%Fix Rate
90%Confidence
100Evidence
2019-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
8 active — — —

Root Cause

Pod cannot be scheduled because all nodes have taints that the pod doesn't tolerate. Common with dedicated node pools (GPU, spot instances), master/control-plane taints, or after cluster upgrades that add taints.

generic

Workarounds

  1. 95% success Add the matching toleration to the pod spec 
    spec:
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"

    Sources: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/

  2. 92% success Check which taints exist on nodes and match tolerations accordingly 
    kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
# Or for a specific node:
kubectl describe node <node-name> | grep -A5 Taints

    Sources: https://kubernetes.io/docs/reference/kubectl/cheatsheet/

Dead Ends

Common approaches that don't work:

  1. Removing taints from all nodes to get the pod scheduled 70% fail

    Taints exist for a reason (e.g., control-plane isolation, GPU reservation). Removing them allows any pod to schedule on specialized nodes, causing resource contention and security issues.

  2. Setting nodeSelector without tolerations 90% fail

    nodeSelector selects nodes by label but doesn't bypass taints. The pod will target the right nodes but still be rejected by their taints.

Error Chain

Frequently confused with: