# Parquet statistics min/max values are incorrect after row group merge

- **ID:** `data/parquet-corrupt-min-max-statistics`
- **Domain:** data
- **Category:** data_error
- **Error Code:** `PARQUET_STATISTICS_CORRUPT`
- **Verification:** ai_generated
- **Fix Rate:** 80%

## Root Cause

When merging Parquet row groups, the min/max statistics are not recomputed from actual data but instead derived from column metadata, leading to incorrect filter pushdown.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| parquet-mr 1.12.0 | active | — | — |
| Spark 3.4.0 | active | — | — |
| Apache Arrow 12.0.0 | active | — | — |

## Workarounds

1. **Use Spark's `OPTIMIZE` command with `REWRITE_DATA` option to force statistics recomputation: `OPTIMUTE table_name REWRITE DATA WHERE partition_column = 'value'`.** (90% success)
   ```
   Use Spark's `OPTIMIZE` command with `REWRITE_DATA` option to force statistics recomputation: `OPTIMUTE table_name REWRITE DATA WHERE partition_column = 'value'`.
   ```
2. **Disable predicate pushdown temporarily by setting `spark.sql.parquet.filterPushdown=false` and re-enable after verifying statistics.** (85% success)
   ```
   Disable predicate pushdown temporarily by setting `spark.sql.parquet.filterPushdown=false` and re-enable after verifying statistics.
   ```

## Dead Ends

- **** — Recomputing statistics by rewriting the entire Parquet file is expensive and may not fix the issue if the metadata is already corrupted. (70% fail)
- **** — Adjusting filter predicates to use more relaxed conditions may bypass the incorrect statistics but leads to performance degradation. (60% fail)