llm api_stability ai_generated partial

LLM API response format changes between calls or after model updates

ID: llm/api-response-format-instability

Also available as: JSON · Markdown
65%Fix Rate
78%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

LLM providers update models silently behind version aliases (e.g., 'gpt-4' points to different snapshots over time). Response formatting, JSON structure, and even reasoning patterns can change without notice.

generic

Workarounds

  1. 92% success Use pinned model versions (date-stamped) in production 
    model='gpt-4-0613'  # pinned, not 'gpt-4' which changes
  2. 90% success Implement robust output parsing with fallbacks and validation 
    try: parse_json(output) except: try: parse_markdown(output) except: raw_text_fallback(output)
  3. 88% success Use structured output mode to decouple format from model behavior 
    response_format with strict schema enforcement is more stable across model updates than free-form output

Dead Ends

Common approaches that don't work:

  1. Parse LLM output with rigid regex or exact string matching 88% fail

    Model updates change phrasing, formatting, and structure. Regex that worked yesterday fails after a silent model update.

  2. Use model alias (e.g., 'gpt-4') for production stability 82% fail

    Aliases like 'gpt-4' are redirected to new snapshots periodically. Use pinned versions like 'gpt-4-0613' for stability.