pytorch autograd_error ai_generated true

RuntimeError: 尝试第二次反向传播，但缓冲已被释放。请在第一次调用 backward 时指定 retain_graph=True。

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

ID: pytorch/grad-accumulation-required-grad-false

其他格式: JSON · Markdown 中文 · English
80%修复率
85%置信度
1证据数
2023-03-15首次发现

版本兼容性

版本状态引入弃用备注
1.10 active — — —
1.11 active — — —
1.12 active — — —
1.13 active — — —
2.0 active — — —
2.1 active — — —
2.2 active — — —
2.3 active — — —

根因分析

在同一个计算图上第二次调用 backward() 而未设置 retain_graph=True，常见于梯度累积循环中重复使用计算图时。

English

Calling backward() twice on the same graph without retain_graph=True, which is common in gradient accumulation loops where the graph is re-used.

generic

官方文档

https://pytorch.org/docs/stable/autograd.html#gradient-accumulation

解决方案

  1. Set retain_graph=True only on the first backward call when accumulating gradients over multiple steps: loss.backward(retain_graph=True) for all but the last step, then loss.backward() on the final step.
  2. Use gradient accumulation with model.zero_grad() and loss.backward() only once per accumulation cycle, ensuring the graph is freed after each step.

无效尝试

常见但无效的做法:

  1. 40% 失败

    Adding retain_graph=True every time hides the real issue of memory growth and can cause OOM.

  2. 50% 失败

    This only works if the graph is re-used; if not, it just defers the error.