pytorch device_error ai_generated true

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu

ID: pytorch/device-mismatch

Also available as: JSON · Markdown
92%Fix Rate
90%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
2 active — — —

Root Cause

Tensors on different devices (CPU vs GPU) used in same operation. Model on GPU but input on CPU or vice versa.

generic

Workarounds

  1. 95% success Use a device variable and .to(device) consistently 
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu'); model.to(device); x = x.to(device)

    Sources: https://pytorch.org/docs/stable/

  2. 90% success Check both model and data are on the same device before forward pass 
    assert next(model.parameters()).device == input_tensor.device
  3. 85% success Move criterion/loss function to device if it has parameters

Dead Ends

Common approaches that don't work:

  1. Move every tensor to GPU at creation time 65% fail

    Some tensors (like labels, indices) should stay on CPU until needed. Eager .cuda() wastes VRAM.

  2. Use .cuda() everywhere instead of .to(device) 72% fail

    .cuda() hardcodes GPU and breaks on CPU-only machines or multi-GPU. Use .to(device) consistently.