elasticsearch mapping_error ai_generated true

illegal_argument_exception: Fielddata is disabled on [message] in [my_index]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting.

ID: elasticsearch/illegal-argument-exception-fielddata

Also available as: JSON · Markdown
90%Fix Rate
90%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
8 active — — —

Root Cause

Attempting to sort or aggregate on a text field. Text fields use inverted index, not doc values needed for these operations.

generic

Workarounds

  1. 95% success Use the .keyword multi-field for aggregations and sorting 
    GET /my_index/_search { 'aggs': { 'top_messages': { 'terms': { 'field': 'message.keyword' }}}}

    Sources: https://www.elastic.co/guide/en/elasticsearch/reference/current/

  2. 85% success Add a keyword sub-field to the mapping and reindex 
    PUT /my_index/_mapping { 'properties': { 'message': { 'type': 'text', 'fields': { 'keyword': { 'type': 'keyword' }}}}}
  3. 80% success Use composite aggregation with keyword field for large cardinality

Dead Ends

Common approaches that don't work:

  1. Enable fielddata=true on the text field 90% fail

    Fielddata on text fields loads all unique terms into JVM heap. Causes massive memory usage and OOM on high-cardinality fields.

  2. Convert the field to keyword type on an existing index 85% fail

    You cannot change field type on an existing index. Requires reindex to a new index with updated mapping.