llm streaming ai_generated true

JSON parse error during streaming — incomplete JSON in stream chunk

ID: llm/streaming-json-partial-parse-error

Also available as: JSON · Markdown
90%Fix Rate
92%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Parsing JSON from streamed LLM output fails because chunks contain incomplete JSON.

generic

Workarounds

  1. 93% success Accumulate stream chunks into a buffer and parse only when complete 
    buffer += chunk; try: result = json.loads(buffer) except: continue

    Sources: https://platform.openai.com/docs/api-reference/streaming

  2. 88% success Use a streaming JSON parser that handles partial input 
    Use libraries like partial-json-parser or ijson

    Sources: https://github.com/promplate/partial-json-parser

Dead Ends

Common approaches that don't work:

  1. JSON.parse each chunk individually 95% fail

    Stream chunks are arbitrary byte boundaries, not JSON boundaries

  2. Wait for a closing brace to trigger parsing 75% fail

    Nested objects have multiple closing braces; triggering on first } gives incomplete parse