llm config_error ai_generated true

ValueError: 当批次大小大于1且使用左侧填充时，必须提供'max_length'或'padding'和'truncation'参数

ValueError: You have to provide either 'max_length' or 'padding' and 'truncation' to use padding side 'left' with batch size > 1

ID: llm/huggingface-tokenizer-padding-side-mismatch

其他格式: JSON · Markdown 中文 · English
85%修复率
88%置信度
1证据数
2023-08-15首次发现

版本兼容性

版本状态引入弃用备注
transformers 4.30.0 active — — —
transformers 4.31.0 active — — —
transformers 4.35.0 active — — —
Python 3.9 active — — —
Python 3.10 active — — —
Python 3.11 active — — —

根因分析

Hugging Face分词器在使用左侧填充（解码器专用LLM常见设置）进行批量推理时，需要显式配置填充和截断参数，否则无法确定序列长度对齐方式。

English

Hugging Face tokenizer requires explicit padding and truncation configuration when using left padding (common for decoder-only LLMs) in batched inference, otherwise it cannot determine the sequence length alignment.

generic

官方文档

https://huggingface.co/docs/transformers/main/en/main_classes/tokenizer#transformers.PreTrainedTokenizer.__call__.padding

解决方案

  1. 在分词前显式配置padding=True, truncation=True和max_length=model_max_length：

tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-hf')
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = 'left'
encoded = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors='pt')
  2. 显式设置tokenizer.pad_token_id，并使用transformers的DataCollatorWithPadding进行动态批处理：

from transformers import DataCollatorWithPadding
data_collator = DataCollatorWithPadding(tokenizer, padding='max_length', max_length=512)

无效尝试

常见但无效的做法:

  1. 70% 失败

    Decoder-only models (e.g., LLaMA, GPT) expect left padding for causal attention masking; right padding causes misalignment and incorrect generation.

  2. 50% 失败

    Bypasses the error but prevents batching, leading to poor throughput and increased latency in production.