kubernetes node_error ai_generated partial

Node not ready: kubelet stopped posting node status

ID: kubernetes/k8s-node-not-ready

Also available as: JSON · Markdown
75%Fix Rate
82%Confidence
70Evidence
2023-06-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
8 active — — —

Root Cause

Node transitions to NotReady when kubelet stops posting status updates, typically due to kubelet crash, networking issues, or resource exhaustion on the node.

generic

Workarounds

  1. 90% success SSH into the node and check kubelet: systemctl status kubelet && journalctl -u kubelet --since '5 min ago' 
    Look for OOM kills, certificate expiry, or container runtime failures in kubelet logs

    Sources: https://kubernetes.io/docs/tasks/debug/debug-cluster/debug-cluster/

  2. 88% success Check node resource pressure: kubectl describe node <name> for MemoryPressure, DiskPressure, PIDPressure conditions 
    kubectl describe node <name> | grep -A5 Conditions

    Sources: https://kubernetes.io/docs/concepts/architecture/nodes/#condition

Dead Ends

Common approaches that don't work:

  1. Force-delete all pods on the node 78% fail

    Pods are a symptom — the node itself is unhealthy and new pods will also fail

  2. Just wait for auto-recovery without investigation 65% fail

    Some causes (disk full, OOM) won't self-heal — need manual intervention

Error Chain

Leads to:
Preceded by:
Frequently confused with: