3 mongodb system_error ai_generated partial

WiredTiger error (3) __wt_file_handle_close, file: WiredTiger.wt, close: No such file or directory

ID: mongodb/wiredtiger-file-close-failed

Also available as: JSON · Markdown · 中文
75%Fix Rate
85%Confidence
1Evidence
2023-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
MongoDB 5.0 active — — —
MongoDB 6.0 active — — —
MongoDB 7.0 active — — —

Root Cause

The WiredTiger storage engine failed to close a file handle, typically due to a filesystem-level corruption or a stale NFS mount causing the underlying file to be deleted while MongoDB still held a reference.

generic

中文

WiredTiger 存储引擎无法关闭文件句柄，通常是由于文件系统级损坏或过时的 NFS 挂载导致底层文件在 MongoDB 仍持有引用时被删除。

Official Documentation

https://www.mongodb.com/docs/manual/tutorial/recover-data-following-unexpected-shutdown/

Workarounds

  1. 70% success Stop mongod, run `mongod --repair` on the data directory, then restart. This rebuilds WiredTiger metadata from remaining data files. 
    Stop mongod, run `mongod --repair` on the data directory, then restart. This rebuilds WiredTiger metadata from remaining data files.
  2. 90% success Restore from a recent backup if repair fails. Use `mongorestore` with the backup dump. 
    Restore from a recent backup if repair fails. Use `mongorestore` with the backup dump.
  3. 85% success If using a replica set, remove the affected node, resync from the primary by clearing the data directory and restarting with `--replSet`. 
    If using a replica set, remove the affected node, resync from the primary by clearing the data directory and restarting with `--replSet`.

中文步骤

  1. 停止 mongod，在数据目录上运行 `mongod --repair`，然后重新启动。这会从剩余的数据文件中重建 WiredTiger 元数据。
  2. 如果修复失败，从最近的备份恢复。使用 `mongorestore` 恢复备份转储。
  3. 如果使用副本集，删除受影响的节点，清空数据目录并用 `--replSet` 重新启动，从主节点重新同步。

Dead Ends

Common approaches that don't work:

  1. 90% fail

    The corruption persists in the WiredTiger metadata; restarting only re-encounters the same error or causes a crash loop.

  2. 95% fail

    This file contains critical storage engine metadata; removing it makes the database unreadable and requires a full resync from replica set or backup.

  3. 80% fail

    Running fsck on a mounted filesystem can cause further corruption; MongoDB requires the data directory to be consistent before repair.