cuda runtime_error ai_generated true

RuntimeError: expected scalar type Half but found Float

ID: cuda/mixed-precision-dtype-error

Also available as: JSON · Markdown
88%Fix Rate
88%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
12 active — — —

Root Cause

Mixed precision training produces tensors with mismatched dtypes (float16 vs float32). Usually a model or loss function not wrapped in autocast.

generic

Workarounds

  1. 92% success Wrap forward pass in torch.cuda.amp.autocast 
    with torch.cuda.amp.autocast(): output = model(input); loss = criterion(output, target)
  2. 88% success Keep loss computation outside autocast for numerical stability 
    Loss scaling and gradient computation should stay in float32
  3. 90% success Use GradScaler to handle float16 gradient underflow 
    scaler = torch.cuda.amp.GradScaler(); scaler.scale(loss).backward(); scaler.step(optimizer)

Dead Ends

Common approaches that don't work:

  1. Cast all tensors to float16 manually 85% fail

    Loss computation and gradient accumulation in float16 causes numerical instability and NaN gradients

  2. Disable mixed precision entirely 70% fail

    Loses 2x memory savings and training speed; not necessary if autocast is used correctly