huggingface config_error ai_generated true

ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Available modules are: ['query', 'value', 'key', 'output']

ID: huggingface/peft-lora-target-modules-mismatch

Also available as: JSON · Markdown · 中文
85%Fix Rate
88%Confidence
1Evidence
2023-06-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
peft>=0.4.0 active — — —
transformers>=4.30.0 active — — —
torch>=1.13.0 active — — —

Root Cause

LoRA target_modules specified with incorrect module names that do not match the base model's actual layer naming convention.

generic

中文

LoRA 的 target_modules 参数指定了错误的模块名称，与基础模型实际的层命名约定不匹配。

Official Documentation

https://huggingface.co/docs/peft/en/developer_guides/lora#target-modules

Workarounds

  1. 90% success Print the model's module names to identify correct target modules: `for name, _ in model.named_modules(): print(name)` then select only linear layers (e.g., 'q_proj', 'v_proj' for LLaMA, 'query', 'value' for BERT). 
    Print the model's module names to identify correct target modules: `for name, _ in model.named_modules(): print(name)` then select only linear layers (e.g., 'q_proj', 'v_proj' for LLaMA, 'query', 'value' for BERT).
  2. 85% success Use peft.get_peft_model() with target_modules='all-linear' if using peft>=0.7.0 and the model is transformer-based, which automatically selects all linear layers except the output projection. 
    Use peft.get_peft_model() with target_modules='all-linear' if using peft>=0.7.0 and the model is transformer-based, which automatically selects all linear layers except the output projection.
  3. 80% success Check the model configuration via model.config.model_type and refer to the PEFT documentation for the correct module names for that architecture. 
    Check the model configuration via model.config.model_type and refer to the PEFT documentation for the correct module names for that architecture.

中文步骤

  1. Print the model's module names to identify correct target modules: `for name, _ in model.named_modules(): print(name)` then select only linear layers (e.g., 'q_proj', 'v_proj' for LLaMA, 'query', 'value' for BERT).
  2. Use peft.get_peft_model() with target_modules='all-linear' if using peft>=0.7.0 and the model is transformer-based, which automatically selects all linear layers except the output projection.
  3. Check the model configuration via model.config.model_type and refer to the PEFT documentation for the correct module names for that architecture.

Dead Ends

Common approaches that don't work:

  1. 50% fail

    The model may not have those exact names; e.g., LLaMA uses 'self_attn.q_proj' while GPT-2 uses 'attn.c_attn'. This can still fail if names are wrong.

  2. 30% fail

    Includes non-linear layers (e.g., activation functions) which are invalid for LoRA and cause shape errors during forward pass.

  3. 40% fail

    'all-linear' was introduced in peft>=0.7.0; older versions raise AttributeError or silently ignore it.