llm parsing_error ai_generated true

json.decoder.JSONDecodeError: Expecting value when parsing LLM output

ID: llm/json-parse-error

Also available as: JSON · Markdown
82%Fix Rate
88%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

LLM output is not valid JSON despite being asked for JSON. Markdown code blocks, trailing text, or truncated output.

generic

Workarounds

  1. 95% success Use structured output / JSON mode if available 
    OpenAI: response_format={'type': 'json_object'}; Anthropic: tool_use for structured output
  2. 88% success Strip markdown code fences before parsing 
    text = re.sub(r'^```json\n?|```$', '', text.strip()); data = json.loads(text)
  3. 82% success Use a lenient JSON parser like json-repair 
    from json_repair import repair_json; data = json.loads(repair_json(text))

Dead Ends

Common approaches that don't work:

  1. Use eval() or ast.literal_eval() to parse LLM output 90% fail

    eval() is a security vulnerability; literal_eval fails on non-Python JSON. Use json.loads.

  2. Prompt harder with ONLY output JSON and nothing else 68% fail

    Prompting alone is unreliable. Models can still add markdown fences or explanatory text.