110 nginx timeout_error ai_generated true

upstream timed out (110: Connection timed out) while reading upstream, client: 10.0.0.1, server: stream.example.com

ID: nginx/proxy-read-timeout-streaming

Also available as: JSON · Markdown · 中文
90%Fix Rate
85%Confidence
1Evidence
2024-03-22First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
nginx/1.18.0 active — — —
nginx/1.20.0 active — — —
nginx/1.24.0 active — — —
nginx/1.26.0 active — — —

Root Cause

proxy_read_timeout is too short for streaming or long-polling responses, causing nginx to close the connection before the upstream finishes sending data.

generic

中文

proxy_read_timeout 对流式或长轮询响应设置过短，导致 nginx 在上游完成发送数据前关闭连接。

Official Documentation

https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout

Workarounds

  1. 90% success Increase proxy_read_timeout to 300s (or higher) in the location or server block: `location /stream { proxy_read_timeout 300s; ... }` 
    Increase proxy_read_timeout to 300s (or higher) in the location or server block: `location /stream { proxy_read_timeout 300s; ... }`
  2. 80% success For streaming endpoints, add a send_timeout directive to avoid client-side timeouts: `send_timeout 300s;` 
    For streaming endpoints, add a send_timeout directive to avoid client-side timeouts: `send_timeout 300s;`
  3. 75% success Implement keepalive on upstream to reduce connection overhead: `upstream backend { server 127.0.0.1:8080; keepalive 32; }` and `proxy_http_version 1.1; proxy_set_header Connection '';` 
    Implement keepalive on upstream to reduce connection overhead: `upstream backend { server 127.0.0.1:8080; keepalive 32; }` and `proxy_http_version 1.1; proxy_set_header Connection '';`

中文步骤

  1. 在 location 或 server 块中将 proxy_read_timeout 增加到 300 秒（或更高）：`location /stream { proxy_read_timeout 300s; ... }`
  2. 对于流式端点，添加 send_timeout 指令以避免客户端超时：`send_timeout 300s;`
  3. 在上游实现 keepalive 以减少连接开销：`upstream backend { server 127.0.0.1:8080; keepalive 32; }` 和 `proxy_http_version 1.1; proxy_set_header Connection '';`

Dead Ends

Common approaches that don't work:

  1. 80% fail

    proxy_connect_timeout controls the initial handshake, not the data transfer phase; the timeout occurs during reading, not connecting.

  2. 40% fail

    Keepalive affects connection reuse, not the per-request timeout for reading response data.

  3. 60% fail

    Infinite timeout can accumulate zombie connections, eventually exhausting worker connections or memory.