# RuntimeError: OutOfMemoryError: Unable to allocate memory for KV cache block. Requested block size: 16, free blocks: 0

- **ID:** `llm/vllm-block-manager-out-of-memory`
- **Domain:** llm
- **Category:** resource_error
- **Verification:** ai_generated
- **Fix Rate:** 80%

## Root Cause

vLLM's PagedAttention block manager exhausts all available GPU memory for KV cache blocks due to insufficient gpu_memory_utilization setting or concurrent requests exceeding memory budget.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| vLLM 0.3.0 | active | — | — |
| vLLM 0.4.0 | active | — | — |
| vLLM 0.4.2 | active | — | — |
| CUDA 12.1 | active | — | — |
| CUDA 12.2 | active | — | — |

## Workarounds

1. **Reduce gpu_memory_utilization to 0.85 or lower and set --max-num-seqs appropriately:

python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-2-7b-hf --gpu-memory-utilization 0.80 --max-num-seqs 64** (90% success)
   ```
   Reduce gpu_memory_utilization to 0.85 or lower and set --max-num-seqs appropriately:

python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-2-7b-hf --gpu-memory-utilization 0.80 --max-num-seqs 64
   ```
2. **Enable swap space via --swap-space to offload KV cache to CPU when GPU memory is tight:

python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-2-7b-hf --swap-space 16** (85% success)
   ```
   Enable swap space via --swap-space to offload KV cache to CPU when GPU memory is tight:

python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-2-7b-hf --swap-space 16
   ```

## Dead Ends

- **** — Larger batch tokens consume more KV cache memory without freeing space; the error persists or worsens. (60% fail)
- **** — Truncates all inputs, degrading model quality for long-context tasks; may still OOM if concurrent requests are high. (40% fail)