# RuntimeError: 在第 500 步损失为 NaN 或 Inf。考虑降低学习率、添加梯度裁剪或检查输入数据中的异常值。

- **ID:** `pytorch/nan-loss-step`
- **领域:** pytorch
- **类别:** assertion_error
- **验证级别:** ai_generated
- **修复率:** 80%

## 根因

训练损失变为 NaN 或 Inf，原因包括梯度爆炸、数值不稳定（例如，对零取对数）或输入数据中存在极端异常值。

## 版本兼容性

| 版本 | 状态 | 引入 | 弃用 |
|------|------|------|------|
| torch>=1.8 | active | — | — |
| torch>=2.0 | active | — | — |

## 解决方案

1. ```
   Add gradient clipping: torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0) after loss.backward() and before optimizer.step()
   ```
2. ```
   Reduce learning rate by factor of 10: optimizer = torch.optim.Adam(model.parameters(), lr=1e-4) instead of 1e-3
   ```
3. ```
   Add a small epsilon to log inputs: loss = F.cross_entropy(logits, targets) inside a try block with torch.clamp(logits, min=-100, max=100)
   ```

## 无效尝试

- **Setting torch.set_default_dtype(torch.float64) to increase precision** — Float64 may delay but not prevent NaN if the root cause is exploding gradients or log(0); also slows training significantly. (65% 失败率)
- **Restarting training from scratch without changes** — The same hyperparameters and data will produce the same NaN at the same step; the issue is deterministic. (95% 失败率)