# RuntimeError: You are trying to train a 4-bit model but you have set `inference_mode=True`. Set `inference_mode=False` for training.

- **ID:** `huggingface/peft-4bit-inference-mode`
- **Domain:** huggingface
- **Category:** config_error
- **Verification:** ai_generated
- **Fix Rate:** 98%

## Root Cause

When using PEFT with a 4-bit quantized base model, the `inference_mode` flag in LoRA configuration must be False for training, but was set to True.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| peft>=0.5.0 | active | — | — |
| transformers>=4.31.0 | active | — | — |
| bitsandbytes>=0.41.0 | active | — | — |

## Workarounds

1. **Set `inference_mode=False` in the LoRA configuration: `lora_config = LoraConfig(r=8, lora_alpha=32, inference_mode=False)`** (98% success)
   ```
   Set `inference_mode=False` in the LoRA configuration: `lora_config = LoraConfig(r=8, lora_alpha=32, inference_mode=False)`
   ```
2. **If using the PEFT model directly, call `model.train()` and ensure the config was created with `inference_mode=False`. Example: `peft_config = LoraConfig.from_pretrained('path')` then modify `peft_config.inference_mode = False` before creating the model.** (95% success)
   ```
   If using the PEFT model directly, call `model.train()` and ensure the config was created with `inference_mode=False`. Example: `peft_config = LoraConfig.from_pretrained('path')` then modify `peft_config.inference_mode = False` before creating the model.
   ```

## Dead Ends

- **** — The error is specifically about the LoRA `inference_mode` parameter, not the model loading quantization. The LoRA config must be explicitly updated. (85% fail)
- **** — The `inference_mode` flag in the LoRA config is separate from the model's training state. `model.train()` does not affect PEFT's internal configuration. (90% fail)