ETRV tensorflow config_error ai_generated true

tensorflow.python.framework.errors_impl.InternalError: TRITONBACKEND_ModelInstanceInitialize: model 'resnet50' version 1 has unsupported TensorFlow runtime version. Expected 2.12.0, got 2.10.0

ID: tensorflow/triton-inference-version-mismatch

Also available as: JSON · Markdown · 中文
88%Fix Rate
85%Confidence
1Evidence
2024-06-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
2.10 active — — —
2.12 active — — —
2.13 active — — —

Root Cause

The TensorFlow runtime version compiled into the Triton Inference Server backend does not match the version required by the saved model.

generic

中文

Triton 推理服务器后端编译的 TensorFlow 运行时版本与保存模型所需的版本不匹配。

Official Documentation

https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/model_configuration.html

Workarounds

  1. 90% success Use a Triton Inference Server Docker image that matches the TensorFlow version of your model. Check the tag at https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver and pull the correct version, e.g., nvcr.io/nvidia/tritonserver:23.12-tf2-py3 for TF 2.12. 
    Use a Triton Inference Server Docker image that matches the TensorFlow version of your model. Check the tag at https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver and pull the correct version, e.g., nvcr.io/nvidia/tritonserver:23.12-tf2-py3 for TF 2.12.
  2. 85% success Export the model using the same TensorFlow version as the Triton backend. Run 'pip install tensorflow==2.12.0' and re-save the model with tf.saved_model.save(). 
    Export the model using the same TensorFlow version as the Triton backend. Run 'pip install tensorflow==2.12.0' and re-save the model with tf.saved_model.save().

中文步骤

  1. Use a Triton Inference Server Docker image that matches the TensorFlow version of your model. Check the tag at https://catalog.ngc.nvidia.com/orgs/nvidia/containers/tritonserver and pull the correct version, e.g., nvcr.io/nvidia/tritonserver:23.12-tf2-py3 for TF 2.12.
  2. Export the model using the same TensorFlow version as the Triton backend. Run 'pip install tensorflow==2.12.0' and re-save the model with tf.saved_model.save().

Dead Ends

Common approaches that don't work:

  1. Reinstall Triton Inference Server with the latest version 50% fail

    The latest Triton may still bundle a different TensorFlow version; you need to match the exact version.

  2. Convert the model to ONNX format 60% fail

    ONNX conversion may not support all TF ops, and the error is about runtime version, not model format.