# ValueError：您传递了 `bnb_4bit_compute_dtype=torch.float16` 的量化配置，但模型权重以 `torch.bfloat16` 加载。两者不兼容。

- **ID:** `huggingface/quantization-dtype-mismatch-bnb`
- **领域:** huggingface
- **类别:** config_error
- **验证级别:** ai_generated
- **修复率:** 90%

## 根因

Bitsandbytes 量化配置的计算数据类型与模型的原始权重数据类型不匹配，导致前向传播时类型转换失败。

## 版本兼容性

| 版本 | 状态 | 引入 | 弃用 |
|------|------|------|------|
| bitsandbytes 0.43.0 | active | — | — |
| transformers 4.44.0 | active | — | — |
| torch 2.3.0 | active | — | — |

## 解决方案

1. ```
   Set `bnb_4bit_compute_dtype=torch.bfloat16` to match the model weight dtype: `quantization_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)`.
   ```
2. ```
   Load model with `torch_dtype=torch.float16` and set `bnb_4bit_compute_dtype=torch.float16`.
   ```

## 无效尝试

- **** — If the model was saved in bfloat16, casting to float16 may cause overflow or underflow; the root mismatch remains if model weights are bfloat16. (50% 失败率)
- **** — Bitsandbytes checks for exact dtype match; this combination will still raise the same error. (100% 失败率)