mongodb system_error ai_generated true

MongoServerError: OplogTruncation: replication recovery failed due to missing oplog entries

ID: mongodb/oplog-truncation-during-recovery

Also available as: JSON · Markdown · 中文
85%Fix Rate
87%Confidence
1Evidence
2023-11-20First Seen

Root Cause

The oplog was truncated (e.g., due to replication lag or manual deletion) before a secondary could replay all entries, causing an irrecoverable gap in the replication stream.

generic

中文

由于复制延迟或手动删除，oplog 在辅助节点重放所有条目之前被截断，导致复制流中出现无法恢复的间隙。

Official Documentation

https://www.mongodb.com/docs/manual/tutorial/resync-replica-set-member/

Workarounds

  1. 95% success Resync the secondary from scratch: stop the secondary, delete its data directory (e.g., rm -rf /data/db/*), then restart and let it perform an initial sync from the primary. 
    Resync the secondary from scratch: stop the secondary, delete its data directory (e.g., rm -rf /data/db/*), then restart and let it perform an initial sync from the primary.
  2. 70% success If the primary still has the missing oplog entries (e.g., in a backup), restore them manually using mongorestore with --oplogReplay, then restart replication. 
    If the primary still has the missing oplog entries (e.g., in a backup), restore them manually using mongorestore with --oplogReplay, then restart replication.

中文步骤

  1. Resync the secondary from scratch: stop the secondary, delete its data directory (e.g., rm -rf /data/db/*), then restart and let it perform an initial sync from the primary.
  2. If the primary still has the missing oplog entries (e.g., in a backup), restore them manually using mongorestore with --oplogReplay, then restart replication.

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Increasing oplog size (e.g., via --oplogSizeMB) prevents future truncation but does not recover the lost entries; the secondary remains stuck.

  2. 90% fail

    Restarting the secondary node only re-attempts the same recovery, which fails again since the oplog gap persists.