llm structured_output ai_generated partial

LLM returns values not in the specified enum when generating structured JSON output

ID: llm/structured-output-hallucinated-enum

Also available as: JSON · Markdown
72%Fix Rate
80%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Even with JSON mode or function calling, LLMs can hallucinate enum values not in the schema. The model 'understands' the schema but doesn't guarantee constraint satisfaction. Only json_schema response_format with strict:true in OpenAI actually enforces enums.

generic

Workarounds

  1. 95% success Use strict: true with json_schema response_format (OpenAI) 
    response_format={'type': 'json_schema', 'json_schema': {'name': 'output', 'strict': True, 'schema': {...}}}
  2. 88% success Validate LLM output against schema and retry on violation 
    for attempt in range(3): output = llm_call(); if validate(output, schema): return output  # retry loop
  3. 78% success Post-process enum fields with fuzzy matching to nearest valid value 
    from difflib import get_close_matches; valid = get_close_matches(output_val, enum_values, n=1)

Dead Ends

Common approaches that don't work:

  1. Define enum in function schema and trust the model to respect it 82% fail

    Function calling schemas are treated as suggestions, not constraints. The model can return any string for an enum field.

  2. Use JSON mode (response_format: json_object) and assume schema compliance 88% fail

    JSON mode only guarantees valid JSON syntax, NOT schema compliance. Fields can be missing, wrong type, or have invalid enum values.