huggingface config_error ai_generated true

ValueError: Truncation side 'left' is not compatible with padding side 'right' for this model. Set truncation_side='right' or padding_side='left'.

ID: huggingface/truncation-side-mismatch-with-padding

Also available as: JSON · Markdown · 中文
93%Fix Rate
89%Confidence
1Evidence
2024-05-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
huggingface/transformers 4.40.0 active — — —
huggingface/tokenizers 0.17.0 active — — —
huggingface/transformers 4.41.0 active — — —

Root Cause

The tokenizer's truncation_side and padding_side are set to opposite sides, causing inconsistent token alignment for decoder-only models.

generic

中文

分词器的 truncation_side 和 padding_side 设置为相反侧，导致仅解码器模型的标记对齐不一致。

Official Documentation

https://huggingface.co/docs/transformers/en/pad_truncation

Workarounds

  1. 95% success Set tokenizer padding_side to 'left' and truncation_side to 'left' for decoder-only models: `tokenizer.padding_side = 'left'; tokenizer.truncation_side = 'left'` before encoding. 
    Set tokenizer padding_side to 'left' and truncation_side to 'left' for decoder-only models: `tokenizer.padding_side = 'left'; tokenizer.truncation_side = 'left'` before encoding.
  2. 85% success Use `tokenizer(text, padding='max_length', truncation=True, return_tensors='pt')` and let the tokenizer auto-resolve by setting `truncation_side='right'` and `padding_side='left'` via `tokenizer.init_kwargs`. 
    Use `tokenizer(text, padding='max_length', truncation=True, return_tensors='pt')` and let the tokenizer auto-resolve by setting `truncation_side='right'` and `padding_side='left'` via `tokenizer.init_kwargs`.

中文步骤

  1. Set tokenizer padding_side to 'left' and truncation_side to 'left' for decoder-only models: `tokenizer.padding_side = 'left'; tokenizer.truncation_side = 'left'` before encoding.
  2. Use `tokenizer(text, padding='max_length', truncation=True, return_tensors='pt')` and let the tokenizer auto-resolve by setting `truncation_side='right'` and `padding_side='left'` via `tokenizer.init_kwargs`.

Dead Ends

Common approaches that don't work:

  1. Set both truncation_side and padding_side to 'right' 70% fail

    For decoder-only models, padding on the right can cause incorrect attention masking; the model expects left padding for generation.

  2. Ignore the warning and proceed with training 100% fail

    This is a ValueError, not a warning; it raises an exception and stops execution.