llm data_error ai_generated true

json.decoder.JSONDecodeError: Expecting property name at position N

ID: llm/structured-output-json-mode-returns-unexpected-field

Also available as: JSON · Markdown · 中文
85%Fix Rate
81%Confidence
1Evidence
2024-11-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
openai>=1.0.0 active — — —
pydantic>=2.0.0 active — — —
python>=3.9 active — — —

Root Cause

When using JSON mode (response_format={type: 'json_object'}), the LLM generates a JSON object with keys not present in the requested schema, causing downstream parsing to fail or validation to reject the output.

generic

中文

使用 JSON 模式（response_format={type: 'json_object'}）时，LLM 生成包含请求模式中不存在的键的 JSON 对象，导致下游解析失败或验证拒绝输出。

Official Documentation

https://platform.openai.com/docs/guides/text-generation/json-mode

Workarounds

  1. 90% success Use Pydantic's 'extra' parameter set to 'forbid' to reject extra fields, and re-prompt the LLM on failure. 
    Use Pydantic's 'extra' parameter set to 'forbid' to reject extra fields, and re-prompt the LLM on failure.
  2. 85% success Post-process the JSON output to filter out unexpected keys before validation. 
    Post-process the JSON output to filter out unexpected keys before validation.

中文步骤

  1. Use Pydantic's 'extra' parameter set to 'forbid' to reject extra fields, and re-prompt the LLM on failure.
  2. Post-process the JSON output to filter out unexpected keys before validation.

Dead Ends

Common approaches that don't work:

  1. Add more examples to the system prompt showing the exact JSON structure 60% fail

    LLMs may still hallucinate extra fields even with examples; examples reduce but don't eliminate the issue.

  2. Use a stronger model like GPT-4 instead of GPT-3.5 50% fail

    While GPT-4 is more reliable, it can still produce extra fields in JSON mode, especially with complex schemas.