kafka network_error ai_generated true

java.io.IOException: Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.

ID: kafka/socket-server-max-too-low

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
1Evidence
2023-05-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Kafka 3.4.0 active — — —
Kafka 3.5.0 active — — —
Kafka 3.6.0 active — — —
Kafka 3.7.0 active — — —

Root Cause

The broker's socket server max connections limit is exhausted because too many concurrent connections from clients exceed the configured value of socket.server.max.connections.

generic

中文

代理的套接字服务器最大连接数限制已耗尽，因为来自客户端的并发连接数超过了配置的 socket.server.max.connections 值。

Official Documentation

https://kafka.apache.org/documentation/#brokerconfigs_socket.server.max.connections

Workarounds

  1. 85% success Increase the socket.server.max.connections configuration in server.properties and restart the broker: socket.server.max.connections=2000 Then monitor connections using: netstat -an | grep :9092 | wc -l 
    Increase the socket.server.max.connections configuration in server.properties and restart the broker:

socket.server.max.connections=2000

Then monitor connections using: netstat -an | grep :9092 | wc -l
  2. 75% success Reduce the number of concurrent connections from clients by implementing connection pooling or batching requests. For example, in a Java consumer, use a single shared KafkaConsumer instance instead of creating per-thread consumers. 
    Reduce the number of concurrent connections from clients by implementing connection pooling or batching requests. For example, in a Java consumer, use a single shared KafkaConsumer instance instead of creating per-thread consumers.

中文步骤

  1. Increase the socket.server.max.connections configuration in server.properties and restart the broker:

socket.server.max.connections=2000

Then monitor connections using: netstat -an | grep :9092 | wc -l
  2. Reduce the number of concurrent connections from clients by implementing connection pooling or batching requests. For example, in a Java consumer, use a single shared KafkaConsumer instance instead of creating per-thread consumers.

Dead Ends

Common approaches that don't work:

  1. Restart the broker 90% fail

    Restarting only temporarily resets connections; the underlying limit is unchanged and will be hit again quickly under high load.

  2. Increase heap memory for the broker 95% fail

    Socket server connections are not directly related to heap memory; the issue is a connection count limit, not memory pressure.