elasticsearch system_error ai_generated partial

OutOfMemoryError: Java heap space: failed to allocate field data for field [user.id]

ID: elasticsearch/field-cache-too-large

Also available as: JSON · Markdown · 中文
72%Fix Rate
87%Confidence
1Evidence
2023-09-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
7.17.0 active — — —
8.10.0 active — — —
8.12.1 active — — —

Root Cause

The fielddata cache for a high-cardinality field (e.g., user.id) consumes too much heap memory, triggering an OutOfMemoryError.

generic

中文

高基数字段（例如user.id）的字段数据缓存消耗过多堆内存，触发内存溢出错误。

Official Documentation

https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html

Workarounds

  1. 85% success Set a fielddata circuit breaker limit: PUT _cluster/settings { "persistent": { "indices.breaker.fielddata.limit": "30%" } } and also set fielddata cache size: PUT _cluster/settings { "persistent": { "indices.fielddata.cache.size": "20%" } } 
    Set a fielddata circuit breaker limit: PUT _cluster/settings { "persistent": { "indices.breaker.fielddata.limit": "30%" } } and also set fielddata cache size: PUT _cluster/settings { "persistent": { "indices.fielddata.cache.size": "20%" } }
  2. 90% success Use doc_values instead of fielddata by reindexing the field with doc_values: true; for keyword fields, doc_values is enabled by default, so avoid aggregating on text fields without doc_values 
    Use doc_values instead of fielddata by reindexing the field with doc_values: true; for keyword fields, doc_values is enabled by default, so avoid aggregating on text fields without doc_values

中文步骤

  1. 设置字段数据断路器限制：PUT _cluster/settings { "persistent": { "indices.breaker.fielddata.limit": "30%" } }，同时设置字段数据缓存大小：PUT _cluster/settings { "persistent": { "indices.fielddata.cache.size": "20%" } }
  2. 使用doc_values代替fielddata：重新索引字段并启用doc_values: true；对于keyword字段，doc_values默认启用，因此避免在没有doc_values的text字段上进行聚合

Dead Ends

Common approaches that don't work:

  1. 75% fail

    Larger heap may postpone OOM but fielddata can grow unbounded; eventually OOM recurs

  2. 80% fail

    Disabling fielddata breaks aggregations and sorting on that field; queries that depend on it will fail