# RuntimeError：期望所有张量在相同设备上，但模型参数已被卸载到CPU

- **ID:** `huggingface/model-offloaded-to-cpu`
- **领域:** huggingface
- **类别:** runtime_error
- **验证级别:** ai_generated
- **修复率:** 80%

## 根因

模型使用device_map='auto'或类似卸载策略加载，但前向传播等操作在没有正确设备放置的情况下在另一个设备上执行。

## 版本兼容性

| 版本 | 状态 | 引入 | 弃用 |
|------|------|------|------|
| transformers>=4.36.0 | active | — | — |
| accelerate>=0.25.0 | active | — | — |
| PyTorch>=2.1.0 | active | — | — |

## 解决方案

1. ```
   Ensure all operations use the same device context. Use accelerate's context manager: from accelerate import dispatch_model; dispatch_model(model, device_map='auto') and then run inference within a with torch.no_grad() block on the same device.
   ```
2. ```
   Set device_map to a specific device (e.g., device_map='cuda:0') instead of 'auto' to force all layers onto GPU if memory permits.
   ```

## 无效尝试

- **Manually moving model to GPU with model.to('cuda') after loading with device_map** — device_map='auto' already handles placement; model.to() overrides it and may cause offloaded layers to be lost or cause memory issues on large models. (70% 失败率)
- **Setting device_map=None and manually placing model on GPU** — For very large models that require offloading, disabling device_map may cause OOM because the entire model won't fit on GPU. (85% 失败率)