cuda runtime_error ai_generated partial

RuntimeError: CUDA error: misaligned address

ID: cuda/misaligned-address

Also available as: JSON · Markdown
55%Fix Rate
78%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
12 active — — —

Root Cause

CUDA kernel accessed memory at an address not aligned to the required boundary. Often caused by custom CUDA kernels or corrupted tensors.

generic

Workarounds

  1. 85% success Set CUDA_LAUNCH_BLOCKING=1 to get the exact kernel and line 
    CUDA_LAUNCH_BLOCKING=1 python train.py
  2. 80% success Check for integer overflow in index calculations for large tensors 
    Use torch.long for indices when tensor dimensions exceed 2^31
  3. 75% success Verify custom CUDA kernel alignment with alignof() and aligned_alloc()

Dead Ends

Common approaches that don't work:

  1. Increase GPU memory allocation 90% fail

    This is an alignment error, not an out-of-memory error; more memory does not fix wrong addresses

  2. Disable CUDA memory caching allocator 82% fail

    The caching allocator does not cause misalignment; disabling it only hurts performance

Error Chain

Frequently confused with: