tensorflow memory_error ai_generated true

ResourceExhaustedError: OOM despite free GPU memory

ID: tensorflow/gpu-memory-fragmentation

Also available as: JSON · Markdown
78%Fix Rate
82%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
2 active — — —

Root Cause

GPU memory fragmented, cant allocate contiguous block.

generic

Workarounds

  1. 88% success Enable memory growth: tf.config.experimental.set_memory_growth(gpu, True)
  2. 85% success Use smaller batch sizes to reduce peak memory fragmentation

Dead Ends

Common approaches that don't work:

  1. Set memory growth to False and pre-allocate all 72% fail

    Other apps cant use GPU

  2. Restart Python process frequently 68% fail

    Loses model state