elasticsearch data_error ai_generated true

IllegalArgumentException: Field [user.name] of type [keyword] in index [logs-2025] has a length of [32768] which exceeds the maximum allowed length of 32766

ID: elasticsearch/field-length-exceeds-keyword-limit

Also available as: JSON · Markdown · 中文
90%Fix Rate
85%Confidence
1Evidence
2024-06-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
7.10.0 active — — —
8.10.0 active — — —
8.13.0 active — — —

Root Cause

A document contains a keyword field value that exceeds the 32766-byte limit, causing indexing rejection.

generic

中文

文档中的关键字字段值超过了 32766 字节的限制，导致索引被拒绝。

Official Documentation

https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html#keyword-field-type-limits

Workarounds

  1. 90% success Change the field type to `text` with a `keyword` subfield for exact matching: `PUT my_index/_mapping { "properties": { "user.name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }`. 
    Change the field type to `text` with a `keyword` subfield for exact matching: `PUT my_index/_mapping { "properties": { "user.name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }`.
  2. 85% success Truncate the field value at the application level before indexing to ensure it stays under 32766 bytes. 
    Truncate the field value at the application level before indexing to ensure it stays under 32766 bytes.
  3. 88% success Use a `keyword` field with `ignore_above` set to a value less than 32766 to skip indexing long values. 
    Use a `keyword` field with `ignore_above` set to a value less than 32766 to skip indexing long values.

中文步骤

  1. 将字段类型更改为 `text` 并添加 `keyword` 子字段用于精确匹配：`PUT my_index/_mapping { "properties": { "user.name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } }`。
  2. 在应用程序级别截断字段值，确保其小于 32766 字节后再索引。
  3. 使用带有 `ignore_above` 设置的 `keyword` 字段，设置值小于 32766 以跳过索引长值。

Dead Ends

Common approaches that don't work:

  1. 80% fail

    `ignore_above` truncates the field, it does not increase the keyword length limit.

  2. 90% fail

    Removing `ignore_above` does not change the hard byte limit for keyword fields.

  3. 85% fail

    Reindexing without addressing the field type will still reject large values.