kubernetes resources ai_generated true

The node was low on resource: ephemeral-storage

ID: kubernetes/pod-evicted

Also available as: JSON · Markdown
91%Fix Rate
93%Confidence
50Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
8 active — — —
8 active — — —

Root Cause

Pod evicted due to node running low on disk (ephemeral storage) or memory.

generic

Workarounds

  1. 92% success Set ephemeral-storage resource limits on containers to prevent excessive disk use 
    resources: limits: ephemeral-storage: 2Gi

    Sources: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

  2. 88% success Clean up container logs and temp files; use emptyDir sizeLimit 
    emptyDir: sizeLimit: 1Gi prevents unbounded temp file growth

Dead Ends

Common approaches that don't work:

  1. Setting very high resource requests to get scheduled on bigger nodes 70% fail

    Wastes cluster resources; doesn't fix the storage consumption

  2. Disabling eviction thresholds on the kubelet 90% fail

    Node can run out of disk and become unresponsive

Error Chain

Frequently confused with: