huggingface config_error ai_generated true

ValueError: repetition_penalty and no_repeat_ngram_size cannot be set simultaneously as they may conflict

ID: huggingface/generation-repetition-penalty-conflict

Also available as: JSON · Markdown · 中文
90%Fix Rate
83%Confidence
1Evidence
2024-02-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
transformers>=4.35.0 active — — —

Root Cause

Both `repetition_penalty` and `no_repeat_ngram_size` are set in the generation config, but their combined effect can produce contradictory constraints on token selection.

generic

中文

在生成配置中同时设置了 `repetition_penalty` 和 `no_repeat_ngram_size`，但它们的组合效果可能对 token 选择产生矛盾约束。

Official Documentation

https://huggingface.co/docs/transformers/en/generation_strategies#repetition-penalty

Workarounds

  1. 95% success Remove one of the conflicting parameters: `model.generate(input_ids, repetition_penalty=1.2) # no no_repeat_ngram_size` or `model.generate(input_ids, no_repeat_ngram_size=3) # no repetition_penalty` 
    Remove one of the conflicting parameters: `model.generate(input_ids, repetition_penalty=1.2)  # no no_repeat_ngram_size` or `model.generate(input_ids, no_repeat_ngram_size=3)  # no repetition_penalty`
  2. 90% success Set the removed parameter to its default value explicitly: `model.generate(input_ids, repetition_penalty=1.0, no_repeat_ngram_size=0)` 
    Set the removed parameter to its default value explicitly: `model.generate(input_ids, repetition_penalty=1.0, no_repeat_ngram_size=0)`
  3. 60% success Use a custom generation config that overrides the conflict check: `from transformers import GenerationConfig; config = GenerationConfig.from_pretrained('model-name', repetition_penalty=1.2, no_repeat_ngram_size=3, _from_model_config=True)` but this may cause undefined behavior. 
    Use a custom generation config that overrides the conflict check: `from transformers import GenerationConfig; config = GenerationConfig.from_pretrained('model-name', repetition_penalty=1.2, no_repeat_ngram_size=3, _from_model_config=True)` but this may cause undefined behavior.

中文步骤

  1. Remove one of the conflicting parameters: `model.generate(input_ids, repetition_penalty=1.2)  # no no_repeat_ngram_size` or `model.generate(input_ids, no_repeat_ngram_size=3)  # no repetition_penalty`
  2. Set the removed parameter to its default value explicitly: `model.generate(input_ids, repetition_penalty=1.0, no_repeat_ngram_size=0)`
  3. Use a custom generation config that overrides the conflict check: `from transformers import GenerationConfig; config = GenerationConfig.from_pretrained('model-name', repetition_penalty=1.2, no_repeat_ngram_size=3, _from_model_config=True)` but this may cause undefined behavior.

Dead Ends

Common approaches that don't work:

  1. Setting both parameters to very low values (e.g., 0.1 and 1) 95% fail

    The conflict check is boolean; any non-default values for both triggers the error, regardless of magnitude.

  2. Ignoring the warning and proceeding with generation 100% fail

    This is a ValueError, not a warning; it halts execution entirely.

  3. Upgrading transformers to 4.40.0 80% fail

    The conflict detection was introduced in 4.35.0; newer versions still enforce the rule.