运行时错误：DataLoader工作进程（pid 12345）收到信号11（段错误）。可能原因：共享内存耗尽、共享内存不足或工作进程过多。
RuntimeError: DataLoader worker (pid 12345) received signal 11 (Segmentation fault). Possible causes: shared memory exhaustion, insufficient shared memory, or too many workers.
ID: pytorch/data-loader-worker-segfault-shared-memory
版本兼容性
| 版本 | 状态 | 引入 | 弃用 | 备注 |
|---|---|---|---|---|
| pytorch>=1.8.0 | active | — | — | — |
| linux>=4.18 | active | — | — | — |
| cuda>=11.0 | active | — | — | — |
根因分析
DataLoader工作进程耗尽系统共享内存（/dev/shm）限制，在尝试将张量从共享内存复制到GPU时导致段错误。
English
DataLoader worker processes exhaust the system's shared memory (/dev/shm) limit, causing a segmentation fault when trying to copy tensors from shared memory to the GPU.
官方文档
https://pytorch.org/docs/stable/data.html#multi-process-data-loading解决方案
-
Increase the system shared memory limit by running the container or process with `--shm-size=8g` (Docker) or `mount -o remount,size=8G /dev/shm` (host). Alternatively, set `DataLoader(pin_memory=False, num_workers=0)` to bypass shared memory entirely.
-
Use `torch.utils.data.DataLoader(..., multiprocessing_context='spawn')` to force the spawn start method, which may reduce shared memory fragmentation on some systems.
无效尝试
常见但无效的做法:
-
Reduce the batch size to lower memory pressure.
70% 失败
Reducing batch size does not directly affect shared memory usage per worker; each worker still allocates its own copy of tensors in shared memory for pin_memory.
-
Increase the number of DataLoader workers to speed up data loading.
90% 失败
Increasing the number of workers exacerbates shared memory exhaustion by spawning more processes that each allocate their own shared memory buffers.