# OSError: Unable to mmap dataset file: [Errno 12] Cannot allocate memory

- **ID:** `huggingface/dataset-memory-mapping-failure`
- **Domain:** huggingface
- **Category:** system_error
- **Error Code:** `ENOMEM`
- **Verification:** ai_generated
- **Fix Rate:** 75%

## Root Cause

The system does not have enough virtual memory address space to memory-map the dataset file, often due to large file size or system ulimit restrictions.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| datasets>=2.10.0 | active | — | — |
| Linux kernel >=5.4 | active | — | — |
| Python>=3.8 | active | — | — |

## Workarounds

1. **Reduce the dataset file size by splitting it into smaller shards before loading. Use `datasets.load_dataset('path/to/dataset', split='train[:50%]')` to load half the data.** (70% success)
   ```
   Reduce the dataset file size by splitting it into smaller shards before loading. Use `datasets.load_dataset('path/to/dataset', split='train[:50%]')` to load half the data.
   ```
2. **Increase the system's vm.max_map_count using `sysctl -w vm.max_map_count=2000000` on Linux. This increases the number of memory-mapped regions allowed.** (85% success)
   ```
   Increase the system's vm.max_map_count using `sysctl -w vm.max_map_count=2000000` on Linux. This increases the number of memory-mapped regions allowed.
   ```
3. **Use `load_dataset(..., streaming=True)` to avoid memory-mapping the entire file by streaming the data in chunks.** (90% success)
   ```
   Use `load_dataset(..., streaming=True)` to avoid memory-mapping the entire file by streaming the data in chunks.
   ```

## Dead Ends

- **** — The error is about virtual memory address space, not physical RAM. Adding RAM does not fix address space exhaustion. (90% fail)
- **** — The error occurs during mmap of the original dataset file, not cache. Disabling caching may cause other issues and does not fix the mmap failure. (80% fail)
- **** — This error is about mmap, not mlock. The ulimit for locked memory is unrelated to virtual memory mapping limits. (95% fail)