llm network_error ai_generated partial

Error: Incomplete stream response - expected more data but connection closed unexpectedly.

ID: llm/truncated-response-in-streaming-mode

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
1Evidence
2023-04-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
openai>=1.0.0 active — — —
httpx>=0.24.0 active — — —

Root Cause

The network connection to the LLM API was interrupted mid-stream due to timeout, server restart, or proxy issues, resulting in a truncated response.

generic

中文

由于超时、服务器重启或代理问题，LLM API的网络连接在流传输过程中中断，导致响应被截断。

Official Documentation

https://platform.openai.com/docs/guides/streaming

Workarounds

  1. 85% success Implement retry logic with exponential backoff for stream interruptions: import time from openai import OpenAI client = OpenAI() max_retries = 3 for attempt in range(max_retries): try: stream = client.chat.completions.create( model='gpt-4', messages=[{'role': 'user', 'content': 'Hello'}], stream=True ) for chunk in stream: print(chunk.choices[0].delta.content or '', end='') break except Exception as e: if 'Incomplete stream' in str(e): wait = 2 ** attempt print(f'Retrying in {wait}s...') time.sleep(wait) else: raise 
    Implement retry logic with exponential backoff for stream interruptions:
import time
from openai import OpenAI

client = OpenAI()
max_retries = 3
for attempt in range(max_retries):
    try:
        stream = client.chat.completions.create(
            model='gpt-4',
            messages=[{'role': 'user', 'content': 'Hello'}],
            stream=True
        )
        for chunk in stream:
            print(chunk.choices[0].delta.content or '', end='')
        break
    except Exception as e:
        if 'Incomplete stream' in str(e):
            wait = 2 ** attempt
            print(f'Retrying in {wait}s...')
            time.sleep(wait)
        else:
            raise
  2. 80% success Use a more robust HTTP client with connection pooling and keep-alive: from httpx import Client, Limits limits = Limits(max_keepalive_connections=5, keepalive_expiry=30.0) with Client(limits=limits) as client: try: response = client.post( 'https://api.openai.com/v1/chat/completions', json={'model': 'gpt-4', 'messages': [{'role': 'user', 'content': 'Hello'}], 'stream': True}, headers={'Authorization': 'Bearer YOUR_API_KEY'}, timeout=30.0 ) for line in response.iter_lines(): if line: print(line) except Exception as e: print(f'Stream error: {e}') 
    Use a more robust HTTP client with connection pooling and keep-alive:
from httpx import Client, Limits

limits = Limits(max_keepalive_connections=5, keepalive_expiry=30.0)
with Client(limits=limits) as client:
    try:
        response = client.post(
            'https://api.openai.com/v1/chat/completions',
            json={'model': 'gpt-4', 'messages': [{'role': 'user', 'content': 'Hello'}], 'stream': True},
            headers={'Authorization': 'Bearer YOUR_API_KEY'},
            timeout=30.0
        )
        for line in response.iter_lines():
            if line:
                print(line)
    except Exception as e:
        print(f'Stream error: {e}')

中文步骤

  1. 为流中断实现带指数退避的重试逻辑：
import time
from openai import OpenAI

client = OpenAI()
max_retries = 3
for attempt in range(max_retries):
    try:
        stream = client.chat.completions.create(
            model='gpt-4',
            messages=[{'role': 'user', 'content': '你好'}],
            stream=True
        )
        for chunk in stream:
            print(chunk.choices[0].delta.content or '', end='')
        break
    except Exception as e:
        if 'Incomplete stream' in str(e):
            wait = 2 ** attempt
            print(f'{wait}秒后重试...')
            time.sleep(wait)
        else:
            raise
  2. 使用更健壮的HTTP客户端，带有连接池和保持活动连接：
from httpx import Client, Limits

limits = Limits(max_keepalive_connections=5, keepalive_expiry=30.0)
with Client(limits=limits) as client:
    try:
        response = client.post(
            'https://api.openai.com/v1/chat/completions',
            json={'model': 'gpt-4', 'messages': [{'role': 'user', 'content': '你好'}], 'stream': True},
            headers={'Authorization': 'Bearer YOUR_API_KEY'},
            timeout=30.0
        )
        for line in response.iter_lines():
            if line:
                print(line)
    except Exception as e:
        print(f'流错误: {e}')

Dead Ends

Common approaches that don't work:

  1. 40% fail

    The error can be caused by server-side issues or network instability, not just client timeout. High timeouts may mask other problems.

  2. 30% fail

    Non-streaming calls may also fail with connection errors, and they don't address the root cause of network instability.