redis network_error ai_generated true

SENTINEL: arbiter node unreachable, failover delayed

ID: redis/sentinel-arbiter-unreachable

Also available as: JSON · Markdown · 中文
88%Fix Rate
87%Confidence
1Evidence
2024-06-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Redis 6.2 active — — —
Redis 7.0 active — — —
Redis 7.2 active — — —

Root Cause

The Sentinel instance cannot communicate with the arbiter node, which is crucial for achieving quorum in failover decisions.

generic

中文

Sentinel 实例无法与仲裁节点通信，而仲裁节点对于故障转移决策中达成法定人数至关重要。

Official Documentation

https://redis.io/docs/latest/operate/oss_and_stack/management/sentinel/

Workarounds

  1. 90% success Check network connectivity to the arbiter node: ping <arbiter-ip> and ensure firewall rules allow Sentinel traffic (port 26379). 
    Check network connectivity to the arbiter node: ping <arbiter-ip> and ensure firewall rules allow Sentinel traffic (port 26379).
  2. 70% success Restart the Sentinel service on the affected node: systemctl restart sentinel (or redis-sentinel restart). 
    Restart the Sentinel service on the affected node: systemctl restart sentinel (or redis-sentinel restart).

中文步骤

  1. 检查与仲裁节点的网络连接：ping <arbiter-ip> 并确保防火墙规则允许 Sentinel 流量（端口 26379）。
  2. 在受影响的节点上重启 Sentinel 服务：systemctl restart sentinel（或 redis-sentinel restart）。

Dead Ends

Common approaches that don't work:

  1. Increase sentinel monitor quorum to 3 to tolerate more failures 40% fail

    Does not fix the network issue; may delay failover further if more nodes are unreachable.

  2. Remove the arbiter node from the Sentinel configuration 50% fail

    Reduces redundancy; if other nodes fail, quorum may not be reached, causing no failover.