TQR tensorflow build_error ai_generated true

ValueError: Quantization range (min, max) not supported for op 'CONV_2D' with input type float32 and output type uint8

ID: tensorflow/tflite-quantization-range

Also available as: JSON · Markdown · 中文
80%Fix Rate
82%Confidence
1Evidence
2024-01-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
tensorflow>=2.12.0 active — — —
tflite>=2.12 active — — —
python>=3.9 active — — —

Root Cause

During TFLite conversion, the quantization parameters (min/max) for a CONV_2D op are either missing or out of the valid range for uint8 quantization, typically because calibration data did not cover the full activation range.

generic

中文

在TFLite转换过程中，CONV_2D操作的量化参数（最小值/最大值）缺失或超出uint8量化的有效范围，通常是因为校准数据未覆盖完整的激活范围。

Official Documentation

https://www.tensorflow.org/lite/performance/post_training_quantization

Workarounds

  1. 80% success Ensure the calibration dataset includes diverse inputs that trigger both minimum and maximum activation values for the CONV_2D layer. For example, use a balanced subset of training data: representative_dataset = lambda: [tf.random.normal([1, 224, 224, 3]) for _ in range(100)] (but with real data). 
    Ensure the calibration dataset includes diverse inputs that trigger both minimum and maximum activation values for the CONV_2D layer. For example, use a balanced subset of training data: representative_dataset = lambda: [tf.random.normal([1, 224, 224, 3]) for _ in range(100)] (but with real data).
  2. 75% success Switch to per-channel quantization or use tf.lite.TFLiteConverter with optimizations=[tf.lite.Optimize.DEFAULT] and specify supported_ops=[tf.lite.OpsSet.TFLITE_BUILTINS_INT8] after ensuring full integer quantization is possible. 
    Switch to per-channel quantization or use tf.lite.TFLiteConverter with optimizations=[tf.lite.Optimize.DEFAULT] and specify supported_ops=[tf.lite.OpsSet.TFLITE_BUILTINS_INT8] after ensuring full integer quantization is possible.

中文步骤

  1. Ensure the calibration dataset includes diverse inputs that trigger both minimum and maximum activation values for the CONV_2D layer. For example, use a balanced subset of training data: representative_dataset = lambda: [tf.random.normal([1, 224, 224, 3]) for _ in range(100)] (but with real data).
  2. Switch to per-channel quantization or use tf.lite.TFLiteConverter with optimizations=[tf.lite.Optimize.DEFAULT] and specify supported_ops=[tf.lite.OpsSet.TFLITE_BUILTINS_INT8] after ensuring full integer quantization is possible.

Dead Ends

Common approaches that don't work:

  1. Increasing the number of calibration steps arbitrarily without verifying data diversity. 70% fail

    More steps of the same data does not improve range coverage; the error persists if the calibration set lacks extreme values.

  2. Manually setting min/max to [-128, 127] in the converter options. 90% fail

    TFLite converter ignores manual override for per-op ranges; it expects calibration to provide them.