# RuntimeError: PEFT adapter weight shape mismatch: expected [4096, 4096] but got [4096, 2048]

- **ID:** `huggingface/peft-adapter-shape-mismatch`
- **Domain:** huggingface
- **Category:** runtime_error
- **Verification:** ai_generated
- **Fix Rate:** 92%

## Root Cause

The PEFT adapter was trained on a model with different hidden dimensions (e.g., a smaller variant) and is being loaded onto a model with incompatible dimensions.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| peft>=0.5.0 | active | — | — |
| transformers>=4.30.0 | active | — | — |
| torch>=1.13.0 | active | — | — |

## Workarounds

1. **Verify the base model used for adapter training: load the correct base model with matching hidden size: from transformers import AutoModel; model = AutoModel.from_pretrained('original-base-model'); model.load_adapter('./adapter_path')** (95% success)
   ```
   Verify the base model used for adapter training: load the correct base model with matching hidden size: from transformers import AutoModel; model = AutoModel.from_pretrained('original-base-model'); model.load_adapter('./adapter_path')
   ```
2. **Check adapter config metadata: print(PeftConfig.from_pretrained('./adapter_path').base_model_name_or_path) to identify the correct base model.** (90% success)
   ```
   Check adapter config metadata: print(PeftConfig.from_pretrained('./adapter_path').base_model_name_or_path) to identify the correct base model.
   ```

## Dead Ends

- **Force load the adapter with `strict=False` to ignore mismatched layers** — The model will silently drop or partially load weights, leading to undefined behavior and poor performance. (90% fail)
- **Manually resize the adapter weights using interpolation** — Adapters are not spatially structured; interpolation can break the learned patterns and cause numerical instability. (85% fail)
- **Set `torch.set_default_dtype(torch.float16)` to avoid shape errors** — Dtype does not affect tensor shape; shape mismatch is a structural issue, not a precision issue. (100% fail)