llm api_error ai_generated partial

openai.APITimeoutError: Request timed out

ID: llm/api-timeout

Also available as: JSON · Markdown
70%Fix Rate
85%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

API request timed out. Long generation, network issues, or API overload.

generic

Workarounds

  1. 88% success Set reasonable timeout with retry logic 
    client = OpenAI(timeout=60.0, max_retries=3)
  2. 85% success Reduce max_tokens to limit generation time 
    Shorter responses generate faster; use max_tokens=1000 instead of 4096
  3. 82% success Use streaming to avoid timeout on long responses 
    stream = client.chat.completions.create(..., stream=True)

Dead Ends

Common approaches that don't work:

  1. Set timeout to very large value (300s+) 65% fail

    Extremely long timeouts tie up resources; if the API is overloaded your request will likely fail anyway

  2. Retry the exact same long prompt immediately 72% fail

    If the prompt caused slow generation (long output), it will timeout again. Reduce max_tokens.