ECF
tensorflow
gpu_error
ai_generated
partial
内部错误：cuDNN执行失败：CUDNN_STATUS_EXECUTION_FAILED
InternalError: cuDNN execution failed: CUDNN_STATUS_EXECUTION_FAILED
ID: tensorflow/cudnn-status-execution-failed
75%修复率
85%置信度
1证据数
2023-08-15首次发现
版本兼容性
| 版本 | 状态 | 引入 | 弃用 | 备注 |
|---|---|---|---|---|
| tensorflow 2.10.0 | active | — | — | — |
| cudnn 8.4.1 | active | — | — | — |
| cuda 11.7 | active | — | — | — |
根因分析
cuDNN遇到执行失败，通常是由于不兼容的张量形状或损坏的GPU状态。
English
cuDNN encountered an execution failure, typically due to incompatible tensor shapes or corrupted GPU state.
官方文档
https://www.tensorflow.org/install/gpu解决方案
-
Reduce batch size to avoid memory pressure: model.fit(..., batch_size=16)
-
Set TF_GPU_ALLOCATOR=cuda_malloc_async to use async allocator: export TF_GPU_ALLOCATOR=cuda_malloc_async
-
Clear GPU memory and reset: tf.keras.backend.clear_session()
无效尝试
常见但无效的做法:
-
60% 失败
Increases batch size thinking more data helps, but often makes shape mismatch worse.
-
30% 失败
Restarting kernel may fix transient state but doesn't address underlying shape issue.