cuda resource_error ai_generated partial

RuntimeError: MPS backend out of memory (MPS allocated: 2.00 GB, limit: 2.00 GB)

ID: cuda/mps-memory-pressure

Also available as: JSON · Markdown · 中文
85%Fix Rate
82%Confidence
1Evidence
2023-12-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
PyTorch 2.1.0 active — — —
macOS 14.0 active — — —
MPS 1.0 active — — —

Root Cause

Apple MPS (Metal Performance Shaders) backend has a fixed memory limit that can be exhausted by large models or batch sizes, even on unified memory systems.

generic

中文

Apple MPS（Metal Performance Shaders）后端有固定的内存限制，即使是在统一内存系统上，大型模型或批量大小也可能耗尽内存。

Official Documentation

https://pytorch.org/docs/stable/mps.html

Workarounds

  1. 85% success Reduce batch size or model size. For example, in PyTorch, set `batch_size = 16` or use gradient accumulation: `optimizer.step()` after multiple `loss.backward()` calls. 
    Reduce batch size or model size. For example, in PyTorch, set `batch_size = 16` or use gradient accumulation: `optimizer.step()` after multiple `loss.backward()` calls.

中文步骤

  1. Reduce batch size or model size. For example, in PyTorch, set `batch_size = 16` or use gradient accumulation: `optimizer.step()` after multiple `loss.backward()` calls.

Dead Ends

Common approaches that don't work:

  1. 90% fail

    The OS enforces a hard limit on MPS memory allocation.

  2. 50% fail

    This is a workaround, not a fix for GPU memory management.