# torch.cuda.OutOfMemoryError：CUDA 内存不足。尝试分配 2.00 GiB。GPU 0 总容量为 79.15 GiB，其中 0 字节空闲。

- **ID:** `llm/vllm-cuda-oom-during-prefill`
- **领域:** llm
- **类别:** resource_error
- **错误码:** `CUDA OOM`
- **验证级别:** ai_generated
- **修复率:** 80%

## 根因

vLLM 的预填充阶段（处理提示）分配与提示长度成比例的 KV 缓存内存；非常长的提示或高批次大小耗尽 GPU 内存，尽管由于碎片化报告可用容量为零。

## 版本兼容性

| 版本 | 状态 | 引入 | 弃用 |
|------|------|------|------|
| vllm==0.4.3 | active | — | — |
| torch==2.3.0 | active | — | — |
| cuda==12.1 | active | — | — |

## 解决方案

1. ```
   Enable automatic prefix caching (`--enable-prefix-caching`) in vLLM to reuse KV cache for common prompt prefixes, reducing memory for repeated prompts.
   ```
2. ```
   Set `max_model_len` to a lower value than the model's maximum (e.g., `--max-model-len 4096`) to limit KV cache allocation per request.
   ```
3. ```
   Use `--enforce-eager` to disable CUDA graphs, which reduces memory fragmentation at the cost of throughput.
   ```

## 无效尝试

- **** — This limits throughput severely and may cause request queuing; the OOM may still occur if a single long prompt exceeds the reduced budget. (75% 失败率)
- **** — vLLM already manages memory; increasing utilization can cause it to allocate more KV cache, worsening fragmentation. (70% 失败率)
- **** — This changes the model entirely and may not be feasible for the application; the OOM can still happen with long prompts on smaller models. (80% 失败率)