llm config_error ai_generated true

openai.BadRequestError: 'response_format' parameter is not supported when using 'tools' parameter

ID: llm/function-call-json-mode-conflict

Also available as: JSON · Markdown · 中文
85%Fix Rate
88%Confidence
1Evidence
2024-04-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
openai>=1.0.0 active — — —
gpt-4-turbo-2024-04-09 active — — —
gpt-4o-2024-05-13 active — — —

Root Cause

OpenAI API enforces mutual exclusivity between structured output via response_format and function/tool calling; both cannot be specified in the same request.

generic

中文

OpenAI API 强制要求通过 response_format 进行结构化输出与函数/工具调用互斥，同一请求中不能同时指定两者。

Official Documentation

https://platform.openai.com/docs/guides/structured-outputs/supported-models

Workarounds

  1. 85% success Use function calling with a single tool that wraps the desired structured output: define a function with the output schema as its parameters, and parse the function call arguments as the structured result. 
    Use function calling with a single tool that wraps the desired structured output: define a function with the output schema as its parameters, and parse the function call arguments as the structured result.
  2. 70% success Use JSON mode with response_format and handle tool calls manually via prompt engineering: include tool descriptions in the system message and parse the LLM's natural language response to extract tool calls. 
    Use JSON mode with response_format and handle tool calls manually via prompt engineering: include tool descriptions in the system message and parse the LLM's natural language response to extract tool calls.
  3. 80% success If using OpenAI Python SDK v1.x, pass tools=None explicitly when using response_format, and manage tool logic separately in a post-processing step. 
    If using OpenAI Python SDK v1.x, pass tools=None explicitly when using response_format, and manage tool logic separately in a post-processing step.

中文步骤

  1. 使用单个工具的函数调用包装所需的结构化输出：定义一个函数，其参数为输出模式，并将函数调用参数解析为结构化结果。
  2. 使用带有 response_format 的 JSON 模式，通过提示工程手动处理工具调用：在系统消息中包含工具描述，并解析 LLM 的自然语言响应以提取工具调用。
  3. 如果使用 OpenAI Python SDK v1.x，在使用 response_format 时显式传递 tools=None，并在后处理步骤中单独管理工具逻辑。

Dead Ends

Common approaches that don't work:

  1. 75% fail

    LLM cannot reliably execute tool calls without explicit tools parameter; output format may not match expected schema

  2. 90% fail

    response_format enforces strict output schema; tool calls have reserved structure that conflicts with arbitrary schemas

  3. 60% fail

    Older API versions may lack the feature entirely or have different buggy behavior; not a sustainable fix