# IngestProcessorException: pipeline [my_pipeline] failed at processor [set] with message [field [user.email] is required]

- **ID:** `elasticsearch/ingest-pipeline-processor-failure`
- **Domain:** elasticsearch
- **Category:** data_error
- **Verification:** ai_generated
- **Fix Rate:** 84%

## Root Cause

An ingest pipeline processor requires a field that is missing in the incoming document, causing the pipeline to abort with a validation error.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| 7.10.0 | active | — | — |
| 7.17.0 | active | — | — |
| 8.0.0 | active | — | — |
| 8.11.0 | active | — | — |

## Workarounds

1. **Modify the pipeline to use an `if` condition or `on_failure` handler to skip missing fields: PUT _ingest/pipeline/my_pipeline { "processors": [ { "set": { "field": "user.email", "value": "{{_ingest._source.user.email}}", "if": "ctx.user?.email != null" } } ] }** (88% success)
   ```
   Modify the pipeline to use an `if` condition or `on_failure` handler to skip missing fields: PUT _ingest/pipeline/my_pipeline { "processors": [ { "set": { "field": "user.email", "value": "{{_ingest._source.user.email}}", "if": "ctx.user?.email != null" } } ] }
   ```
2. **Add a default value processor before the failing one to ensure the field exists: { "set": { "field": "user.email", "value": "unknown@example.com", "override": false } }** (82% success)
   ```
   Add a default value processor before the failing one to ensure the field exists: { "set": { "field": "user.email", "value": "unknown@example.com", "override": false } }
   ```
3. **Use the `script` processor to conditionally set the field: { "script": { "source": "if (ctx.user == null) { ctx.user = [:]; } if (ctx.user.email == null) { ctx.user.email = 'fallback@example.com'; }" } }** (85% success)
   ```
   Use the `script` processor to conditionally set the field: { "script": { "source": "if (ctx.user == null) { ctx.user = [:]; } if (ctx.user.email == null) { ctx.user.email = 'fallback@example.com'; }" } }
   ```

## Dead Ends

- **** — This removes all pipeline processing, which may break other data transformations or enrichments that are needed. (70% fail)
- **** — This pollutes the data with meaningless values and may cause downstream issues in queries or aggregations. (60% fail)
- **** — The error is by design; upgrading does not change the validation logic for required fields. (90% fail)