kafka performance_error ai_generated true

Log compaction lagging: dirty ratio exceeded

ID: kafka/compaction-lag

Also available as: JSON · Markdown
78%Fix Rate
82%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
3 active — — —

Root Cause

Log compaction falling behind, increasing disk usage.

generic

Workarounds

  1. 88% success Increase log.cleaner.threads and log.cleaner.dedupe.buffer.size
  2. 85% success Monitor compaction lag: kafka.log.LogCleanerManager metrics

Dead Ends

Common approaches that don't work:

  1. Disable log compaction 82% fail

    Unbounded disk growth

  2. Set min.cleanable.dirty.ratio to 0 78% fail

    Excessive CPU from constant compaction