CUDNN_STATUS_NOT_INITIALIZED pytorch runtime_error ai_generated true

RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED when calling cudnnCreate

ID: pytorch/cudnn-status-not-initialized

Also available as: JSON · Markdown · 中文
80%Fix Rate
87%Confidence
1Evidence
2023-01-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
pytorch>=1.10 active — — —
cuda>=11.0 active — — —
cudnn>=8.0 active — — —

Root Cause

cuDNN library failed to initialize, often due to incompatible CUDA/cuDNN versions, missing library files, or corrupted installation.

generic

中文

cuDNN 库初始化失败，通常由于 CUDA/cuDNN 版本不兼容、缺少库文件或安装损坏。

Official Documentation

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html

Workarounds

  1. 90% success Verify CUDA and cuDNN versions match PyTorch requirements: run `python -c "import torch; print(torch.version.cuda); print(torch.backends.cudnn.version())"` and compare with PyTorch documentation. 
    Verify CUDA and cuDNN versions match PyTorch requirements: run `python -c "import torch; print(torch.version.cuda); print(torch.backends.cudnn.version())"` and compare with PyTorch documentation.
  2. 85% success Reinstall cuDNN by downloading the correct version from NVIDIA and copying library files to the CUDA toolkit directory (e.g., /usr/local/cuda/lib64). 
    Reinstall cuDNN by downloading the correct version from NVIDIA and copying library files to the CUDA toolkit directory (e.g., /usr/local/cuda/lib64).
  3. 80% success Use a PyTorch Docker image with pre-matched CUDA/cuDNN versions: `docker pull pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel` 
    Use a PyTorch Docker image with pre-matched CUDA/cuDNN versions: `docker pull pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel`

中文步骤

  1. Verify CUDA and cuDNN versions match PyTorch requirements: run `python -c "import torch; print(torch.version.cuda); print(torch.backends.cudnn.version())"` and compare with PyTorch documentation.
  2. Reinstall cuDNN by downloading the correct version from NVIDIA and copying library files to the CUDA toolkit directory (e.g., /usr/local/cuda/lib64).
  3. Use a PyTorch Docker image with pre-matched CUDA/cuDNN versions: `docker pull pytorch/pytorch:2.0.1-cuda11.7-cudnn8-devel`

Dead Ends

Common approaches that don't work:

  1. Reinstalling PyTorch without changing CUDA version 85% fail

    If the underlying CUDA/cuDNN mismatch persists, reinstalling PyTorch alone does not resolve it.

  2. Setting environment variable CUDNN_LOGINFO_DBG=1 90% fail

    This only enables logging, does not fix the initialization issue.

  3. Downgrading PyTorch to an older version 70% fail

    May temporarily work but is not a proper fix; the root cause is version compatibility.