llm input_error ai_generated true

InvalidRequestError: total tokens exceed model maximum

ID: llm/token-limit-exceeded

Also available as: JSON · Markdown
85%Fix Rate
88%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Combined input+output tokens exceed model limit.

generic

Workarounds

  1. 92% success Count tokens before request with tiktoken/tokenizer
  2. 90% success Reserve token budget: max_tokens = model_limit - input_tokens - buffer

Dead Ends

Common approaches that don't work:

  1. Set max_tokens to model limit ignoring input 78% fail

    Input takes from same budget

  2. Remove system prompt to save tokens 80% fail

    Loses instruction context