pytorch type_error ai_generated true

RuntimeError: MPS backend does not support float64. Falling back to float32. Please cast your tensors to float32 explicitly

ID: pytorch/mps-float64-fallback

Also available as: JSON · Markdown · 中文
90%Fix Rate
87%Confidence
1Evidence
2023-03-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
torch>=1.12.0 active — — —
macOS>=12.3 active — — —
MPS>=1.0 active — — —

Root Cause

Apple MPS (Metal Performance Shaders) backend lacks support for float64 (double precision) tensors, causing operations to fail or silently fall back to float32, which may lead to precision loss or type mismatches.

generic

中文

Apple MPS（Metal Performance Shaders）后端不支持 float64（双精度）张量，导致操作失败或静默回退到 float32，可能造成精度损失或类型不匹配。

Official Documentation

https://pytorch.org/docs/stable/notes/mps.html

Workarounds

  1. 95% success Explicitly cast all input tensors to float32 before moving to MPS: tensor = tensor.float().to('mps'). For models, use model = model.float().to('mps'). 
    Explicitly cast all input tensors to float32 before moving to MPS: tensor = tensor.float().to('mps'). For models, use model = model.float().to('mps').
  2. 90% success Override the default dtype at the start of your script: torch.set_default_dtype(torch.float32). Also ensure all data loaders yield float32 tensors by adding a transform: transforms.ConvertImageDtype(torch.float32). 
    Override the default dtype at the start of your script: torch.set_default_dtype(torch.float32). Also ensure all data loaders yield float32 tensors by adding a transform: transforms.ConvertImageDtype(torch.float32).

中文步骤

  1. Explicitly cast all input tensors to float32 before moving to MPS: tensor = tensor.float().to('mps'). For models, use model = model.float().to('mps').
  2. Override the default dtype at the start of your script: torch.set_default_dtype(torch.float32). Also ensure all data loaders yield float32 tensors by adding a transform: transforms.ConvertImageDtype(torch.float32).

Dead Ends

Common approaches that don't work:

  1. 60% fail

    Tensors already created as float64 remain float64; only newly created tensors are affected.

  2. 70% fail

    Float16 may underflow or overflow in training, leading to NaN loss or convergence issues.