huggingface runtime_error ai_generated partial

运行时错误：PEFT模型与动态适配器不兼容torch.compile

RuntimeError: torch.compile is not supported for PEFT models with dynamic adapters

ID: huggingface/torch-compile-incompatible-with-peft

其他格式: JSON · Markdown 中文 · English
80%修复率
85%置信度
1证据数
2024-03-15首次发现

版本兼容性

版本状态引入弃用备注
transformers>=4.38.0 active — — —
peft>=0.9.0 active — — —
torch>=2.2.0 active — — —

根因分析

当适配器被动态加载或切换时，torch.compile在PEFT模型上失败，因为计算图在运行时发生变化。

English

torch.compile fails on PEFT models when adapters are dynamically loaded or switched because the computation graph changes during runtime.

generic

官方文档

https://huggingface.co/docs/peft/main/en/developer_guides/torch_compile

解决方案

  1. Use `torch.compile` only after loading all adapters and disable dynamic switching: `model = torch.compile(model, dynamic=False)`
  2. Avoid torch.compile for PEFT models with dynamic adapters; use eager mode with `model.eval()` and manual fusion instead.
  3. Set environment variable `TORCH_COMPILE_DISABLE=1` and rely on `torch.inference_mode()` for inference.

无效尝试

常见但无效的做法:

  1. 95% 失败

    The issue is not about optimization mode; dynamic adapter switching breaks torch.compile's graph tracing regardless of mode.

  2. 90% 失败

    The incompatibility is fundamental to PEFT's dynamic nature, not a torch bug; nightly versions have the same limitation.

  3. 85% 失败

    torch.compile captures the graph at compile time; gradient disabling does not prevent graph changes from dynamic adapters.