llm input_error ai_generated true

openai.BadRequestError: function_call arguments must be valid JSON - truncated input detected

ID: llm/function-call-argument-truncation

Also available as: JSON · Markdown · 中文
85%Fix Rate
88%Confidence
1Evidence
2024-02-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
openai==1.14.0 active — — —
openai==1.20.0 active — — —
gpt-4-0125-preview active — — —
gpt-3.5-turbo-0125 active — — —

Root Cause

When using function calling, the LLM generates argument JSON that exceeds the model's context window, causing it to truncate mid-JSON, resulting in malformed arguments.

generic

中文

使用函数调用时，LLM 生成的参数 JSON 超过了模型的上下文窗口，导致其在 JSON 中间被截断，产生格式错误的参数。

Official Documentation

https://platform.openai.com/docs/guides/function-calling

Workarounds

  1. 80% success Reduce the size of the function description or parameters in the function schema to fit within the context window. 
    Reduce the size of the function description or parameters in the function schema to fit within the context window.
  2. 85% success Split the function call into multiple smaller calls, each with fewer arguments. 
    Split the function call into multiple smaller calls, each with fewer arguments.
  3. 90% success Use a model with a larger context window (e.g., gpt-4-32k or gpt-4-turbo) for complex function calls. 
    Use a model with a larger context window (e.g., gpt-4-32k or gpt-4-turbo) for complex function calls.

中文步骤

  1. 减小函数描述或参数的大小，使其适应上下文窗口。
  2. 将函数调用拆分为多个较小的调用，每个调用使用更少的参数。
  3. 使用具有更大上下文窗口的模型（例如 gpt-4-32k 或 gpt-4-turbo）进行复杂函数调用。

Dead Ends

Common approaches that don't work:

  1. 70% fail

    The issue is context window exhaustion, not response token limit; max_tokens only caps the output length, not the input.

  2. 95% fail

    The truncation is deterministic given the same input; retrying won't change the behavior.