ETQU tensorflow build_error ai_generated partial

ConverterError: Quantization not supported for op 'Cumsum'. Op has no registered quantized kernel.

ID: tensorflow/tflite-converter-unsupported-op

Also available as: JSON · Markdown · 中文
80%Fix Rate
86%Confidence
1Evidence
2023-05-18First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
TensorFlow 2.7.0 active — — —
TFLite 2.7.0 active — — —

Root Cause

During TFLite conversion with quantization enabled, an operation (e.g., Cumsum, Gather, or TopK) does not have a quantized kernel implementation, causing the converter to fail.

generic

中文

在启用量化的 TFLite 转换过程中，某个操作（例如 Cumsum、Gather 或 TopK）没有量化的内核实现，导致转换器失败。

Official Documentation

https://www.tensorflow.org/lite/performance/post_training_quantization#full_integer_quantization

Workarounds

  1. 85% success Use selective quantization: apply quantization only to supported ops by using `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]` and exclude unsupported ops by falling back to float. Alternatively, use `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]` to allow select TensorFlow ops. 
    Use selective quantization: apply quantization only to supported ops by using `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]` and exclude unsupported ops by falling back to float. Alternatively, use `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]` to allow select TensorFlow ops.
  2. 80% success If the unsupported op is critical, convert the model without quantization (set optimizations=[]) and then apply post-training quantization using a representative dataset that avoids the op, or use full-integer quantization with fallback. 
    If the unsupported op is critical, convert the model without quantization (set optimizations=[]) and then apply post-training quantization using a representative dataset that avoids the op, or use full-integer quantization with fallback.
  3. 70% success Rewrite the model to avoid the unsupported op. For example, replace tf.math.cumsum with a loop or a different algorithm that uses supported ops like Add and Scan. 
    Rewrite the model to avoid the unsupported op. For example, replace tf.math.cumsum with a loop or a different algorithm that uses supported ops like Add and Scan.

中文步骤

  1. 使用选择性量化：仅对支持的操作应用量化，通过设置 `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]`，并对不支持的操作回退到浮点。或者使用 `converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]` 允许选择 TensorFlow 操作。
  2. 如果不受支持的操作是关键性的，则在不量化的情况下转换模型（设置 optimizations=[]），然后使用避免该操作的代表性数据集应用训练后量化，或使用带有回退的全整数量化。
  3. 重写模型以避免不受支持的操作。例如，用循环或使用支持的操作（如 Add 和 Scan）的不同算法替换 tf.math.cumsum。

Dead Ends

Common approaches that don't work:

  1. 60% fail

    This may avoid the quantization error but results in a non-optimized model, losing the benefits of quantization (size reduction, speed).

  2. 80% fail

    Manual replacement is error-prone and may change model behavior; also, the custom op may not be supported in TFLite either.

  3. 50% fail

    Even in the latest versions, some ops still lack quantized kernels; upgrading alone does not guarantee support.