communication alerting ai_generated true

Unique alerts grouped into existing incident — PagerDuty grouping too aggressive

ID: communication/pagerduty-alert-grouping-swallows-unique

Also available as: JSON · Markdown
80%Fix Rate
85%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

PagerDuty intelligent grouping merges distinct alerts into one incident.

generic

Workarounds

  1. 88% success Use content-based alert grouping with specific dedup_key patterns 
    Set dedup_key to include service + error_type

    Sources: https://support.pagerduty.com/docs/intelligent-alert-grouping

  2. 82% success Configure time-based grouping with appropriate window 
    Set 5-minute window; review grouped incidents weekly to tune

    Sources: https://support.pagerduty.com/docs/intelligent-alert-grouping

Dead Ends

Common approaches that don't work:

  1. Disable alert grouping entirely 70% fail

    Without grouping, alert storms create hundreds of incidents

  2. Set grouping window to very short (1 minute) 65% fail

    Short windows still group rapid-fire alerts