TORCH_SCRIPT_TRACE_DYNAMIC_CF pytorch runtime_error ai_generated partial

RuntimeError: TracerWarning: Using a tensor as a Python bool might cause the trace to be incorrect. We can't record the control flow of this loop.

ID: pytorch/jit-trace-dynamic-control-flow

Also available as: JSON · Markdown · 中文
75%Fix Rate
83%Confidence
1Evidence
2023-05-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
torch>=1.0 active — — —
torch>=1.8 for improved warnings active — — —

Root Cause

TorchScript tracing encountered a data-dependent control flow (e.g., if tensor > 0), which cannot be captured statically; tracing converts it to a constant.

generic

中文

TorchScript跟踪遇到了数据相关的控制流（例如，if tensor > 0），无法静态捕获；跟踪将其转换为常量。

Official Documentation

https://pytorch.org/docs/stable/jit.html#tracing-vs-scripting

Workarounds

  1. 85% success Replace dynamic control flow with torch.where or masked operations: instead of 'if x > 0: y = a else: y = b', use 'y = torch.where(x > 0, a, b)'. This keeps the graph static. 
    Replace dynamic control flow with torch.where or masked operations: instead of 'if x > 0: y = a else: y = b', use 'y = torch.where(x > 0, a, b)'. This keeps the graph static.
  2. 80% success Use torch.jit.script with type annotations: @torch.jit.script; def forward(self, x: torch.Tensor) -> torch.Tensor: ... This supports dynamic control flow. 
    Use torch.jit.script with type annotations: @torch.jit.script; def forward(self, x: torch.Tensor) -> torch.Tensor: ... This supports dynamic control flow.

中文步骤

  1. 用torch.where或掩码操作替换动态控制流：不用'if x > 0: y = a else: y = b'，而用'y = torch.where(x > 0, a, b)'。这保持图静态。
  2. 使用带类型注解的torch.jit.script：@torch.jit.script; def forward(self, x: torch.Tensor) -> torch.Tensor: ... 这支持动态控制流。

Dead Ends

Common approaches that don't work:

  1. @torch.jit.script 60% fail

    Using @torch.jit.script instead of tracing may still fail if the function uses dynamic shapes; scripting requires full type annotations.

  2. torch.jit._state.disable_tracing() 90% fail

    Suppressing the warning with torch.jit._state.disable_tracing() hides the issue but the exported model will produce wrong results for different inputs.