# RuntimeError: Loss is NaN or Inf at step 500. Consider reducing learning rate, adding gradient clipping, or checking input data for outliers.

- **ID:** `pytorch/nan-loss-step`
- **Domain:** pytorch
- **Category:** assertion_error
- **Verification:** ai_generated
- **Fix Rate:** 80%

## Root Cause

Training loss becomes NaN or Inf due to exploding gradients, numerical instability (e.g., log of zero), or corrupted input data with extreme values.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| torch>=1.8 | active | — | — |
| torch>=2.0 | active | — | — |

## Workarounds

1. **Add gradient clipping: torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0) after loss.backward() and before optimizer.step()** (85% success)
   ```
   Add gradient clipping: torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0) after loss.backward() and before optimizer.step()
   ```
2. **Reduce learning rate by factor of 10: optimizer = torch.optim.Adam(model.parameters(), lr=1e-4) instead of 1e-3** (75% success)
   ```
   Reduce learning rate by factor of 10: optimizer = torch.optim.Adam(model.parameters(), lr=1e-4) instead of 1e-3
   ```
3. **Add a small epsilon to log inputs: loss = F.cross_entropy(logits, targets) inside a try block with torch.clamp(logits, min=-100, max=100)** (80% success)
   ```
   Add a small epsilon to log inputs: loss = F.cross_entropy(logits, targets) inside a try block with torch.clamp(logits, min=-100, max=100)
   ```

## Dead Ends

- **Setting torch.set_default_dtype(torch.float64) to increase precision** — Float64 may delay but not prevent NaN if the root cause is exploding gradients or log(0); also slows training significantly. (65% fail)
- **Restarting training from scratch without changes** — The same hyperparameters and data will produce the same NaN at the same step; the issue is deterministic. (95% fail)