llm data_error ai_generated true

InvalidRequestError：function_call参数必须是有效的JSON — 流式模式检测到格式错误的JSON

InvalidRequestError: function_call arguments must be valid JSON — streaming mode detected malformed JSON

ID: llm/function-call-json-schema-violation-in-streaming

其他格式: JSON · Markdown 中文 · English
85%修复率
88%置信度
1证据数
2024-02-10首次发现

版本兼容性

版本状态引入弃用备注
openai==1.14.0 active — — —
anthropic==0.28.0 active — — —
gpt-4-0613 active — — —
claude-3-sonnet-20240229 active — — —

根因分析

当流式传输函数调用时，LLM可能会在中间数据块中发出不完整或格式错误的JSON，如果验证严格，会导致API拒绝请求。

English

When streaming function calls, the LLM may emit incomplete or malformed JSON in intermediate chunks, causing the API to reject the request if validation is strict.

generic

官方文档

https://platform.openai.com/docs/guides/function-calling

解决方案

  1. 累积流式数据块，仅在接收完最后一个数据块后解析JSON（例如：`function_call_chunks = []; for chunk in stream: if chunk.choices[0].delta.function_call: function_call_chunks.append(chunk.choices[0].delta.function_call.arguments); full_json = ''.join(function_call_chunks); args = json.loads(full_json)`）。
  2. 使用像`json-repair`这样的JSON修复库，在验证前修复来自流式数据块的格式错误的JSON。

无效尝试

常见但无效的做法:

  1. 70% 失败

    Disabling streaming entirely (stream=False) avoids the issue but defeats the purpose of real-time interaction.

  2. 90% 失败

    Manually escaping JSON characters in the function schema doesn't help because the error is in the LLM output, not the schema.

  3. 80% 失败

    Increasing temperature or top_p to force more varied output doesn't fix JSON structure issues.