llm memory_error ai_generated true

OutOfMemoryError: cannot load model weights

ID: llm/model-loading-oom

Also available as: JSON · Markdown
80%Fix Rate
84%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Model weights exceed available RAM/VRAM.

generic

Workarounds

  1. 90% success Use quantization: load_in_4bit or load_in_8bit
  2. 88% success Use device_map="auto" for multi-GPU model parallelism

Dead Ends

Common approaches that don't work:

  1. Use swap space for model loading 75% fail

    Extremely slow inference

  2. Load model partially and ignore missing weights 88% fail

    Broken model, garbage output