python system_error ai_generated partial

内存错误：无法分配形状为 (100000000,) 且数据类型为 float64 的数组

MemoryError: Unable to allocate array with shape (100000000,) and data type float64

ID: python/starlette-memory-error-upload

其他格式: JSON · Markdown 中文 · English
80%修复率
82%置信度
0证据数
2024-05-11首次发现

版本兼容性

版本状态引入弃用备注
3.x active — — —

根因分析

在 Starlette/FastAPI 端点中尝试一次性将大文件上传读入内存。

English

Attempting to read a large file upload into memory all at once in a Starlette/FastAPI endpoint.

generic

解决方案

  1. 85% 成功率 
    Stream the file using UploadFile.read(chunk_size) in a loop and process chunks: while chunk := await file.read(1024*1024): process(chunk)
  2. 80% 成功率 
    Use a temporary file on disk: with tempfile.NamedTemporaryFile(delete=False) as tmp: shutil.copyfileobj(file.file, tmp)

无效尝试

常见但无效的做法:

  1. 90% 失败

    These don't affect Python's heap allocation for large numpy arrays; physical RAM is exhausted.

  2. 80% 失败

    This loads the whole file into a bytes object, still causing memory pressure.