database replication_error ai_generated true

Seconds_Behind_Master: 3600 (or Seconds_Behind_Source: 3600)

ID: database/mysql-replication-lag

Also available as: JSON · Markdown
80%Fix Rate
83%Confidence
50Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
8 active

Root Cause

The MySQL replica is significantly behind the primary in processing binary log events. Replication lag means reads from the replica return stale data. Caused by slow single-threaded SQL applier, large transactions, insufficient replica hardware, or DDL operations on the replica.

generic

Workarounds

  1. 85% success Enable parallel replication to use multiple applier threads
    Set on the replica: SET GLOBAL replica_parallel_workers = 8; SET GLOBAL replica_parallel_type = 'LOGICAL_CLOCK'; SET GLOBAL replica_preserve_commit_order = ON; This allows transactions that were executed in parallel on the primary to be applied in parallel on the replica. Requires MySQL 8.0.27+ for best results.
  2. 80% success Identify and optimize the slow queries or large transactions causing lag
    Check the current replay position: SHOW REPLICA STATUS\G; Look at Relay_Log_File and Relay_Log_Pos. Examine what's being applied: SELECT * FROM performance_schema.replication_applier_status_by_worker; Break large transactions into smaller batches on the primary. Avoid DDL during peak traffic.

Dead Ends

Common approaches that don't work:

  1. Stopping and restarting replication to reset the lag counter 90% fail

    Stopping replication does not reduce the backlog of events to apply. When replication restarts, it resumes from where it stopped and the lag remains. During the stop period, even more events accumulate on the primary, increasing the lag.

  2. Adding more replicas to distribute the read load without fixing the lag 75% fail

    All replicas receive the same binary log stream. If one replica cannot keep up, additional replicas with similar hardware will have the same lag. Adding replicas increases primary network overhead without fixing the processing bottleneck.

Error Chain

Leads to:
Preceded by:
Frequently confused with: