huggingface config_error ai_generated true

UserWarning: Using `pad_token_id` but `padding_side` is not set. The tokenizer will use 'right' padding by default, which may not be optimal for decoder-only models.

ID: huggingface/tokenizer-fast-vs-slow-padding

Also available as: JSON · Markdown · 中文
95%Fix Rate
82%Confidence
1Evidence
2023-11-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
transformers>=4.34.0 active — — —
tokenizers>=0.14.0 active — — —

Root Cause

When a tokenizer's `padding_side` is not explicitly set, it defaults to 'right', which causes incorrect causal masking in decoder-only models like GPT or LLaMA, leading to degraded generation quality.

generic

中文

当分词器的 `padding_side` 未显式设置时，默认为 'right'，这会导致仅解码器模型（如 GPT 或 LLaMA）中的因果掩码不正确，从而降低生成质量。

Official Documentation

https://huggingface.co/docs/transformers/pad_truncation#padding-and-truncation

Workarounds

  1. 95% success Set `tokenizer.padding_side = 'left'` before tokenization for decoder-only models. This ensures padding tokens are on the left and do not interfere with causal attention. 
    Set `tokenizer.padding_side = 'left'` before tokenization for decoder-only models. This ensures padding tokens are on the left and do not interfere with causal attention.
  2. 85% success Use the `pad_token_id` and explicitly pass `attention_mask` to the model to avoid reliance on padding_side defaults. 
    Use the `pad_token_id` and explicitly pass `attention_mask` to the model to avoid reliance on padding_side defaults.

中文步骤

  1. Set `tokenizer.padding_side = 'left'` before tokenization for decoder-only models. This ensures padding tokens are on the left and do not interfere with causal attention.
  2. Use the `pad_token_id` and explicitly pass `attention_mask` to the model to avoid reliance on padding_side defaults.

Dead Ends

Common approaches that don't work:

  1. 60% fail

    Setting `padding_side='right'` explicitly silences the warning but does not fix the causal masking issue for decoder-only models.

  2. 40% fail

    Ignoring the warning and proceeding with training often leads to subtle quality degradation that is hard to detect until evaluation.