pytorch autograd_error ai_generated true

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

ID: pytorch/grad-accumulation-required-grad-false

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
1Evidence
2023-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
1.10 active — — —
1.11 active — — —
1.12 active — — —
1.13 active — — —
2.0 active — — —
2.1 active — — —
2.2 active — — —
2.3 active — — —

Root Cause

Calling backward() twice on the same graph without retain_graph=True, which is common in gradient accumulation loops where the graph is re-used.

generic

中文

在同一个计算图上第二次调用 backward() 而未设置 retain_graph=True，常见于梯度累积循环中重复使用计算图时。

Official Documentation

https://pytorch.org/docs/stable/autograd.html#gradient-accumulation

Workarounds

  1. 80% success Set retain_graph=True only on the first backward call when accumulating gradients over multiple steps: loss.backward(retain_graph=True) for all but the last step, then loss.backward() on the final step. 
    Set retain_graph=True only on the first backward call when accumulating gradients over multiple steps: loss.backward(retain_graph=True) for all but the last step, then loss.backward() on the final step.
  2. 90% success Use gradient accumulation with model.zero_grad() and loss.backward() only once per accumulation cycle, ensuring the graph is freed after each step. 
    Use gradient accumulation with model.zero_grad() and loss.backward() only once per accumulation cycle, ensuring the graph is freed after each step.

中文步骤

  1. Set retain_graph=True only on the first backward call when accumulating gradients over multiple steps: loss.backward(retain_graph=True) for all but the last step, then loss.backward() on the final step.
  2. Use gradient accumulation with model.zero_grad() and loss.backward() only once per accumulation cycle, ensuring the graph is freed after each step.

Dead Ends

Common approaches that don't work:

  1. 40% fail

    Adding retain_graph=True every time hides the real issue of memory growth and can cause OOM.

  2. 50% fail

    This only works if the graph is re-used; if not, it just defers the error.