llm network_error ai_generated partial

Error: 不完整的流式响应 - 预期更多数据但连接意外关闭。

Error: Incomplete stream response - expected more data but connection closed unexpectedly.

ID: llm/truncated-response-in-streaming-mode

其他格式: JSON · Markdown 中文 · English
80%修复率
85%置信度
1证据数
2023-04-10首次发现

版本兼容性

版本状态引入弃用备注
openai>=1.0.0 active
httpx>=0.24.0 active

根因分析

由于超时、服务器重启或代理问题,LLM API的网络连接在流传输过程中中断,导致响应被截断。

English

The network connection to the LLM API was interrupted mid-stream due to timeout, server restart, or proxy issues, resulting in a truncated response.

generic

官方文档

https://platform.openai.com/docs/guides/streaming

解决方案

  1. 为流中断实现带指数退避的重试逻辑:
    import time
    from openai import OpenAI
    
    client = OpenAI()
    max_retries = 3
    for attempt in range(max_retries):
        try:
            stream = client.chat.completions.create(
                model='gpt-4',
                messages=[{'role': 'user', 'content': '你好'}],
                stream=True
            )
            for chunk in stream:
                print(chunk.choices[0].delta.content or '', end='')
            break
        except Exception as e:
            if 'Incomplete stream' in str(e):
                wait = 2 ** attempt
                print(f'{wait}秒后重试...')
                time.sleep(wait)
            else:
                raise
  2. 使用更健壮的HTTP客户端,带有连接池和保持活动连接:
    from httpx import Client, Limits
    
    limits = Limits(max_keepalive_connections=5, keepalive_expiry=30.0)
    with Client(limits=limits) as client:
        try:
            response = client.post(
                'https://api.openai.com/v1/chat/completions',
                json={'model': 'gpt-4', 'messages': [{'role': 'user', 'content': '你好'}], 'stream': True},
                headers={'Authorization': 'Bearer YOUR_API_KEY'},
                timeout=30.0
            )
            for line in response.iter_lines():
                if line:
                    print(line)
        except Exception as e:
            print(f'流错误: {e}')

无效尝试

常见但无效的做法:

  1. 40% 失败

    The error can be caused by server-side issues or network instability, not just client timeout. High timeouts may mask other problems.

  2. 30% 失败

    Non-streaming calls may also fail with connection errors, and they don't address the root cause of network instability.