elasticsearch resource_error ai_generated true

TranslogTooLargeException: translog size [2.5gb] exceeds maximum size [1.0gb]

ID: elasticsearch/translog-too-large

Also available as: JSON · Markdown · 中文
80%Fix Rate
84%Confidence
1Evidence
2024-04-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
elasticsearch 8.11 active — — —
elasticsearch 8.12 active — — —
elasticsearch 7.17 active — — —
opensearch 2.9 active — — —

Root Cause

The translog has grown beyond the configured maximum size (default 1GB), often due to slow flushing or high write rate.

generic

中文

事务日志增长超过配置的最大大小（默认1GB），通常由于刷新缓慢或高写入速率。

Official Documentation

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html

Workarounds

  1. 85% success Manually flush the index to clear translog: POST my_index/_flush 
    Manually flush the index to clear translog: POST my_index/_flush
  2. 80% success Increase translog threshold and perform flush: PUT my_index/_settings { "index.translog.flush_threshold_size": "2gb", "index.translog.sync_interval": "5s" } 
    Increase translog threshold and perform flush: PUT my_index/_settings { "index.translog.flush_threshold_size": "2gb", "index.translog.sync_interval": "5s" }
  3. 75% success Reduce indexing rate temporarily to allow translog to flush naturally, then increase flush threshold permanently. 
    Reduce indexing rate temporarily to allow translog to flush naturally, then increase flush threshold permanently.

中文步骤

  1. 手动刷新索引以清除事务日志。
  2. 增加事务日志阈值并执行刷新。
  3. 暂时降低索引速率以允许事务日志自然刷新，然后永久增加刷新阈值。

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Corrupts the index; Elasticsearch will fail to recover and may lose data.

  2. 50% fail

    Delays flush but does not reduce existing translog size; may cause OOM on large translog.

  3. 80% fail

    Force merge does not affect translog size; translog is separate from segments.