AttributeError: 'Tensor' object has no attribute 'grad'
ID: python/attributeerror-tensor-no-grad
Version Compatibility
| Version | Status | Introduced | Deprecated | Notes |
|---|---|---|---|---|
| 2 | active | — | — | — |
| 2 | active | — | — | — |
Root Cause
The tensor's .grad attribute is None (or inaccessible) because requires_grad was not set to True before the forward pass and backward() call. In PyTorch 2.1+, accessing .grad on a tensor that never participated in gradient computation raises AttributeError. The fix is to ensure requires_grad=True is set at tensor creation time.
genericWorkarounds
-
92% success Set requires_grad=True on the tensor before the forward pass
When creating the tensor: 'x = torch.tensor([1.0, 2.0], requires_grad=True)'. For existing tensors: 'x.requires_grad_(True)' (in-place) or 'x = x.clone().requires_grad_(True)'. This must be done before the forward computation, not after backward().
Sources: https://pytorch.org/docs/stable/autograd.html#locally-disabling-gradient-computation
-
80% success Use torch.no_grad() context correctly (only where intended)
Ensure the forward pass that should compute gradients is NOT wrapped in 'with torch.no_grad():'. This context manager disables gradient tracking for all operations within it. Use it only for inference or evaluation, not during training. Check for global torch.set_grad_enabled(False) calls as well.
Sources: https://pytorch.org/docs/stable/generated/torch.no_grad.html
-
75% success Verify the tensor is a leaf tensor and part of the computation graph
Check 'tensor.requires_grad' (should be True), 'tensor.is_leaf' (should be True for parameter tensors), and 'tensor.grad_fn' (should be None for leaf tensors, non-None for computed tensors). After backward(), only leaf tensors with requires_grad=True have .grad populated.
Sources: https://pytorch.org/docs/stable/notes/autograd.html
Dead Ends
Common approaches that don't work:
-
Calling .backward() again on the loss
88% fail
Calling backward() a second time without retain_graph=True raises 'RuntimeError: Trying to backward through the graph a second time'. Even with retain_graph=True, if the tensor never had requires_grad=True, no gradient will be computed for it regardless of how many times backward() is called.
-
Using .detach() on the tensor before accessing .grad
95% fail
detach() creates a new tensor that is explicitly disconnected from the computation graph. A detached tensor never has gradients. This is the opposite of what is needed; it removes gradient tracking rather than enabling it.
-
Wrapping the access in a try/except and defaulting to zero
85% fail
Silently replacing a missing gradient with zeros masks a real bug in the training pipeline. The model will appear to train but the affected parameters will never update, leading to degraded or nonsensical model performance that is extremely difficult to debug.