pytorch memory_error ai_generated true

RuntimeError: DataLoader worker (pid 12345) is killed by signal: Killed

ID: pytorch/dataloader-worker-killed

Also available as: JSON · Markdown

82%Fix Rate

88%Confidence

3Evidence

2023-01-01First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
2	active	—	—	—
2	active	—	—	—

DataLoader worker process killed by OS OOM killer. Too many workers, large prefetch, or shared memory limit in Docker.

generic

90% success Reduce num_workers and prefetch_factor
```
DataLoader(dataset, num_workers=2, prefetch_factor=2)  # default prefetch_factor=2
```
Sources: https://pytorch.org/docs/stable/
88% success In Docker: increase shared memory size
```
docker run --shm-size=8g ... # or --ipc=host
```
82% success Use persistent_workers=True to avoid respawning overhead
```
DataLoader(dataset, num_workers=4, persistent_workers=True)
```

Common approaches that don't work:

Set num_workers=0 permanently 60% fail
Fixes the crash but kills data loading performance; training becomes I/O bound
Increase system swap space 72% fail
Swap on GPU training machines causes massive slowdowns; fix the memory usage instead