python system_error ai_generated true

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 15.3 GiB for an array with shape (45000, 45000) and data type float64

ID: python/numpy-memory-allocation

Also available as: JSON · Markdown · 中文
80%Fix Rate
87%Confidence
0Evidence
2024-09-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3.x active — — —

Root Cause

Requested array exceeds available RAM; often from dense distance matrices or cross joins.

generic

中文

请求的数组超过可用内存；常由稠密距离矩阵或笛卡尔积引起。

Workarounds

  1. 85% success 
    Use float32: np.zeros((n, m), dtype=np.float32)
  2. 90% success 
    Chunk with sklearn.metrics.pairwise_distances_chunked or use sparse matrices

Dead Ends

Common approaches that don't work:

  1. 85% fail

    Frees only a few GB; array still too large

  2. 70% fail

    Massive slowdown, may still OOM