ERR redis resource_error ai_generated true

ERR I/O threads pool exhausted: no available thread for connection

ID: redis/io-threads-pool-starvation

Also available as: JSON · Markdown · 中文
80%Fix Rate
82%Confidence
1Evidence
2023-11-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
6.2 active — — —
7.0 active — — —
7.2 active — — —

Root Cause

Redis's I/O threads configuration has too few threads to handle the current connection load, leading to thread pool starvation.

generic

中文

Redis 的 I/O 线程配置中线程数过少，无法处理当前连接负载，导致线程池耗尽。

Official Documentation

https://redis.io/docs/latest/operate/oss_admin/io-threads/

Workarounds

  1. 85% success Increase the number of I/O threads in redis.conf: io-threads 8 (adjust based on CPU cores) and restart Redis. 
    Increase the number of I/O threads in redis.conf: io-threads 8 (adjust based on CPU cores) and restart Redis.
  2. 80% success Reduce the number of concurrent connections by implementing connection pooling in client applications, e.g., using a pool with max 50 connections per client. 
    Reduce the number of concurrent connections by implementing connection pooling in client applications, e.g., using a pool with max 50 connections per client.
  3. 75% success Enable I/O thread read/write by setting io-threads-do-reads yes (if not already enabled) to distribute load more evenly. 
    Enable I/O thread read/write by setting io-threads-do-reads yes (if not already enabled) to distribute load more evenly.

中文步骤

  1. Increase the number of I/O threads in redis.conf: io-threads 8 (adjust based on CPU cores) and restart Redis.
  2. Reduce the number of concurrent connections by implementing connection pooling in client applications, e.g., using a pool with max 50 connections per client.
  3. Enable I/O thread read/write by setting io-threads-do-reads yes (if not already enabled) to distribute load more evenly.

Dead Ends

Common approaches that don't work:

  1. Increase maxclients to allow more connections. 70% fail

    This does not increase the number of I/O threads; it only allows more connections to queue up, worsening the starvation.

  2. Disable I/O threads entirely by setting io-threads to 1. 50% fail

    This removes the thread pool but shifts all I/O to the main thread, potentially causing CPU bottlenecks and higher latency.

  3. Restart Redis to clear the thread pool state. 80% fail

    The pool exhaustion is due to configuration and load, not a transient state; restarting only provides temporary relief until the same load returns.