database resource_error ai_generated true

MongoServerError: WiredTiger cache is full

ID: database/mongo-wiredtiger-cache-full

Also available as: JSON · Markdown
82%Fix Rate
85%Confidence
45Evidence
2021-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
7 active — — —

Root Cause

MongoDB's WiredTiger storage engine cache has exceeded its configured or effective limit and the eviction threads cannot free pages fast enough. This causes new operations to stall or fail. Common causes: storage.wiredTiger.engineConfig.cacheSizeGB set too low, long-running transactions holding dirty pages pinned, large write bursts overwhelming eviction, or the working set exceeding available memory. The default cache size is 50% of (RAM - 1GB).

generic

Workarounds

  1. 88% success Increase cacheSizeGB to an appropriate value based on available RAM and working set 
    Check current cache usage:
  db.serverStatus().wiredTiger.cache

Look at 'bytes currently in the cache' vs 'maximum bytes configured'.
Set cache to ~60% of RAM minus 1GB (leave room for OS):

  # In mongod.conf:
  storage:
    wiredTiger:
      engineConfig:
        cacheSizeGB: 12  # For a 24GB host

  # Or at runtime (MongoDB 4.0.12+):
  db.adminCommand({setParameter: 1, wiredTigerEngineRuntimeConfig: 'cache_size=12G'})

    Sources: https://www.mongodb.com/docs/manual/reference/configuration-options/#mongodb-setting-storage.wiredTiger.engineConfig.cacheSizeGB

  2. 82% success Identify and kill long-running transactions pinning dirty cache pages 
    Find long-running operations:
  db.currentOp({"active": true, "secs_running": {"$gt": 30}})

Check transaction state:
  db.currentOp({"$all": true}).inprog.filter(op => op.transaction)

Kill problematic operations:
  db.killOp(<opid>)

Set a default transaction timeout to prevent future occurrences:
  db.adminCommand({setParameter: 1, transactionLifetimeLimitSeconds: 60})

    Sources: https://www.mongodb.com/docs/manual/core/transactions-production-consideration/

  3. 75% success Tune WiredTiger eviction settings to handle write-heavy workloads 
    Adjust eviction triggers to start cleaning earlier:

  # In mongod.conf:
  storage:
    wiredTiger:
      engineConfig:
        configString: "eviction_target=75,eviction_trigger=95,eviction_dirty_target=5,eviction_dirty_trigger=20,eviction=(threads_min=4,threads_max=8)"

Monitor eviction metrics:
  db.serverStatus().wiredTiger.cache['tracked dirty bytes in the cache']
  db.serverStatus().wiredTiger.cache['pages evicted by application threads']  # Should be 0; nonzero means eviction is overwhelmed

    Sources: https://source.wiredtiger.com/develop/tune_cache.html

Dead Ends

Common approaches that don't work:

  1. Set cacheSizeGB to the entire system RAM 80% fail

    MongoDB shares the host with the OS page cache, journaling, connections, and aggregation pipelines. Setting WiredTiger cache to total RAM starves the OS file cache (needed for reading compressed data from disk) and other MongoDB subsystems, causing OOM kills. MongoDB docs recommend no more than 50% of RAM minus 1GB for the WiredTiger cache.

  2. Restart mongod to clear the cache without investigating the cause 75% fail

    A restart clears the cache temporarily but the issue recurs immediately once the workload resumes. After restart, the cache is cold so performance is initially worse due to cache warming. If the root cause is undersized cache, a write burst, or long-running transactions, the problem returns within minutes to hours.

Error Chain

Leads to:
Preceded by:
Frequently confused with: