huggingface type_error ai_generated true

RuntimeError: expected Float but got Half during training

ID: huggingface/dtype-mismatch-training

Also available as: JSON · Markdown
82%Fix Rate
86%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
4 active — — —

Root Cause

Training dtype conflict between model and optimizer.

generic

Workarounds

  1. 92% success Use Trainer with fp16=True or bf16=True for proper mixed precision
  2. 88% success Ensure loss scaling is enabled: TrainingArguments(fp16=True)

Dead Ends

Common approaches that don't work:

  1. Cast all parameters to float32 72% fail

    Doubles memory usage

  2. Disable mixed precision 68% fail

    Slower training