pytorch config_error ai_generated true

RuntimeError: parameter group's learning rate is not a float or a tensor of scalar type

ID: pytorch/tensor-requires-grad-and-optimizer-param-group

Also available as: JSON · Markdown · 中文
75%Fix Rate
82%Confidence
1Evidence
2023-08-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
1.10 active — — —
1.11 active — — —
1.12 active — — —
1.13 active — — —
2.0 active — — —
2.1 active — — —
2.2 active — — —
2.3 active — — —

Root Cause

Passing a non-float or non-scalar tensor as learning rate in optimizer parameter groups, often due to using a list or array instead of a single value.

generic

中文

在优化器参数组中将非浮点数或非标量张量作为学习率传递，通常是因为使用了列表或数组而不是单个值。

Official Documentation

https://pytorch.org/docs/stable/optim.html#per-parameter-options

Workarounds

  1. 85% success Ensure each parameter group's 'lr' is a single float or a torch scalar tensor. For example, when creating optimizer with per-parameter options, use a float value directly. 
    Ensure each parameter group's 'lr' is a single float or a torch scalar tensor. For example, when creating optimizer with per-parameter options, use a float value directly.
  2. 80% success If using a learning rate scheduler that returns a tensor, convert it to a float before assigning to param_groups. 
    If using a learning rate scheduler that returns a tensor, convert it to a float before assigning to param_groups.

中文步骤

  1. Ensure each parameter group's 'lr' is a single float or a torch scalar tensor. For example, when creating optimizer with per-parameter options, use a float value directly.
  2. If using a learning rate scheduler that returns a tensor, convert it to a float before assigning to param_groups.

Dead Ends

Common approaches that don't work:

  1. 40% fail

    This only changes the data type but doesn't fix the root cause if the value is still a collection.

  2. 60% fail

    This may hide the error but leads to unexpected behavior if the list is not reduced.