# ValueError: You passed `quantization_config` with `bnb_4bit_compute_dtype=torch.float16` but the model weights are loaded in torch.float32. This may cause numerical instability.

- **ID:** `huggingface/quantization-config-dtype-mismatch`
- **Domain:** huggingface
- **Category:** config_error
- **Verification:** ai_generated
- **Fix Rate:** 88%

## Root Cause

Mismatch between the compute dtype specified in quantization config and the actual model weight dtype leads to potential numerical issues.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| transformers>=4.30.0 | active | — | — |
| bitsandbytes>=0.41.0 | active | — | — |

## Workarounds

1. **Set `bnb_4bit_compute_dtype` to match the model weight dtype: `quantization_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=model.dtype)`** (95% success)
   ```
   Set `bnb_4bit_compute_dtype` to match the model weight dtype: `quantization_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=model.dtype)`
   ```
2. **Load the model with the correct torch_dtype: `model = AutoModelForCausalLM.from_pretrained('model-name', torch_dtype=torch.float16, quantization_config=quantization_config)`** (90% success)
   ```
   Load the model with the correct torch_dtype: `model = AutoModelForCausalLM.from_pretrained('model-name', torch_dtype=torch.float16, quantization_config=quantization_config)`
   ```

## Dead Ends

- **Setting `bnb_4bit_compute_dtype` to `torch.float32` without changing model weights** — This only silences the warning but does not fix the underlying dtype mismatch if the model is in float16. (70% fail)
- **Ignoring the warning and proceeding with training** — Numerical instability can lead to NaN loss or diverging gradients, especially in mixed-precision training. (60% fail)