llm encoding_error ai_generated true

json.decoder.JSONDecodeError: 在函数调用参数流中，第 1 行第 1023 列（字符 1022）处字符串未终止

json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1023 (char 1022) in function call arguments stream

ID: llm/function-call-arguments-truncated-in-stream

其他格式: JSON · Markdown 中文 · English
85%修复率
90%置信度
1证据数
2024-01-10首次发现

版本兼容性

版本状态引入弃用备注
openai-python>=1.0.0 active — — —
gpt-4-0613 active — — —
gpt-3.5-turbo-0613 active — — —

根因分析

当流式传输函数调用时，参数作为 JSON 字符串发送，可能被分割到多个块中，导致过早解析时 JSON 不完整。

English

When streaming function calls, the arguments are sent as a JSON string that may be split across multiple chunks, causing incomplete JSON when parsed prematurely.

generic

官方文档

https://platform.openai.com/docs/guides/function-calling/streaming-function-calls

解决方案

  1. Buffer all function call arguments chunks until a complete JSON can be parsed. Example: `buffer = ""; for chunk in response: if chunk.choices[0].delta.function_call.arguments: buffer += chunk.choices[0].delta.function_call.arguments; try: args = json.loads(buffer); break; except JSONDecodeError: continue`
  2. Use the OpenAI library's built-in function call handling which automatically accumulates arguments: `tool_calls = chunk.choices[0].delta.tool_calls` and use `accumulated_arguments[tool_call_index] += chunk.arguments`.
  3. Set `stream_options={"include_usage": True}` to get a final chunk with complete function call info, though this may not always include full arguments.

无效尝试

常见但无效的做法:

  1. 80% 失败

    Max_tokens affects the total output length, not the chunking behavior; stream chunks are inherently arbitrary.

  2. 40% 失败

    This works but defeats the purpose of streaming for user experience; also, it may not be feasible for long-running calls.

  3. 60% 失败

    This is actually a valid approach but requires careful buffering; the dead end is when developers try to parse each chunk individually.