huggingface config_error ai_generated true

ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Available modules: ['query', 'value', 'key', 'output']

ID: huggingface/lora-target-modules-mismatch

Also available as: JSON · Markdown · 中文
85%Fix Rate
90%Confidence
1Evidence
2024-02-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
peft>=0.7.0 active — — —
transformers>=4.35.0 active — — —

Root Cause

The LoRA configuration specifies target module names that do not match the actual naming convention of the base model's linear layers, often due to model architecture differences (e.g., LLaMA vs GPT-2).

generic

中文

LoRA配置指定的目标模块名称与基础模型线性层的实际命名约定不匹配，通常是由于模型架构差异（例如LLaMA vs GPT-2）。

Official Documentation

https://huggingface.co/docs/peft/en/developer_guides/lora#target-modules

Workarounds

  1. 95% success Print the model's module names to find correct targets: for name, module in model.named_modules(): if 'linear' in str(type(module)).lower(): print(name). Then set target_modules to the discovered names. 
    Print the model's module names to find correct targets: for name, module in model.named_modules(): if 'linear' in str(type(module)).lower(): print(name). Then set target_modules to the discovered names.
  2. 80% success Use 'all-linear' as target_modules if supported by PEFT version, but verify with a small test run first. 
    Use 'all-linear' as target_modules if supported by PEFT version, but verify with a small test run first.

中文步骤

  1. Print the model's module names to find correct targets: for name, module in model.named_modules(): if 'linear' in str(type(module)).lower(): print(name). Then set target_modules to the discovered names.
  2. Use 'all-linear' as target_modules if supported by PEFT version, but verify with a small test run first.

Dead Ends

Common approaches that don't work:

  1. Guessing module names based on other models (e.g., using 'q_proj' for a model that uses 'query') 80% fail

    Module names are model-specific; guessing leads to repeated mismatches. Must inspect the model's actual layer names.

  2. Setting target_modules to 'all-linear' without checking compatibility 40% fail

    While 'all-linear' often works, it may include modules that are not suitable for LoRA (e.g., output projection in some architectures), causing training instability or poor performance.