huggingface config_error ai_generated true

值错误：`max_new_tokens` (1000) 与输入长度 (512) 之和超过了模型的最大位置嵌入 (2048)。请减少 `max_new_tokens` 或截断输入。

ValueError: The sum of `max_new_tokens` (1000) and input length (512) exceeds the model's maximum position embeddings (2048). Reduce `max_new_tokens` or truncate input.

ID: huggingface/text-generation-pipeline-max-new-tokens-exceeded

其他格式: JSON · Markdown 中文 · English
93%修复率
89%置信度
1证据数
2023-10-01首次发现

版本兼容性

版本状态引入弃用备注
transformers>=4.38.0 active — — —
torch>=2.0.0 active — — —
Python>=3.8 active — — —

根因分析

总序列长度（输入标记数 + 要生成的新标记数）超过了模型的最大位置嵌入大小，这是位置编码的硬性限制。

English

The total sequence length (input tokens + new tokens to generate) exceeds the model's maximum position embedding size, which is a hard limit for positional encodings.

generic

官方文档

https://huggingface.co/docs/transformers/en/main_classes/text_generation#transformers.GenerationConfig.max_new_tokens

解决方案

  1. Reduce max_new_tokens so that input_length + max_new_tokens <= model_max_length:
from transformers import pipeline
generator = pipeline('text-generation', model='gpt2')
model_max_length = generator.model.config.max_position_embeddings  # 1024 for gpt2
input_length = len(generator.tokenizer(prompt)['input_ids'])
max_new_tokens = min(500, model_max_length - input_length - 10)  # safety margin
result = generator(prompt, max_new_tokens=max_new_tokens)
  2. Truncate the input to allow more generation tokens:
truncated_prompt = generator.tokenizer.decode(generator.tokenizer(prompt, truncation=True, max_length=512)['input_ids'])
result = generator(truncated_prompt, max_new_tokens=1000)

无效尝试

常见但无效的做法:

  1. 100% 失败

    Setting max_new_tokens to a very large value (e.g., 10000) expecting the model to handle it, which causes the same error.

  2. 50% 失败

    Using max_length instead of max_new_tokens may truncate input silently and produce incomplete outputs.