ros2 network_error ai_generated partial

[WARN] [rmw_fastrtps]: Discovery timed out after 30 seconds, some nodes may not be discovered

ID: ros2/ros2-graph-discovery-delay-large-system

Also available as: JSON · Markdown · 中文
90%Fix Rate
87%Confidence
1Evidence
2024-05-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
ros2:humble active
ros2:iron active
rmw_fastrtps_cpp:6.3.0 active
fastdds:2.10.0 active

Root Cause

Fast DDS discovery phase exceeded the default timeout due to a large number of nodes (e.g., >20) on the same network, causing incomplete ROS graph discovery.

generic

中文

Fast DDS 发现阶段因同一网络上节点数量过多(例如 >20)而超过默认超时,导致 ROS 图发现不完整。

Official Documentation

https://fast-dds.docs.eprosima.com/en/latest/fastdds/ros2/discovery_server.html

Workarounds

  1. 90% success Use Fast DDS discovery server to reduce network traffic. Create a discovery server: fastdds discovery --server-id 0 Then launch nodes with the server environment: ROS_DISCOVERY_SERVER=127.0.0.1:11811 ros2 run my_package my_node This centralizes discovery and reduces timeout issues.
    Use Fast DDS discovery server to reduce network traffic. Create a discovery server:
    fastdds discovery --server-id 0
    
    Then launch nodes with the server environment:
    ROS_DISCOVERY_SERVER=127.0.0.1:11811 ros2 run my_package my_node
    
    This centralizes discovery and reduces timeout issues.
  2. 80% success Increase the discovery timeout and reduce discovery traffic by setting environment variables: export FASTRTPS_DEFAULT_PROFILES_FILE=/path/to/fastdds_profile.xml In the XML, set: <discovery_config> <leaseDuration>PT60S</leaseDuration> <announcementPeriod>PT5S</announcementPeriod> </discovery_config> Then restart nodes.
    Increase the discovery timeout and reduce discovery traffic by setting environment variables:
    export FASTRTPS_DEFAULT_PROFILES_FILE=/path/to/fastdds_profile.xml
    
    In the XML, set:
    <discovery_config>
      <leaseDuration>PT60S</leaseDuration>
      <announcementPeriod>PT5S</announcementPeriod>
    </discovery_config>
    
    Then restart nodes.

中文步骤

  1. 使用 Fast DDS 发现服务器减少网络流量。创建发现服务器:
    fastdds discovery --server-id 0
    
    然后使用服务器环境启动节点:
    ROS_DISCOVERY_SERVER=127.0.0.1:11811 ros2 run my_package my_node
    
    这集中了发现过程并减少了超时问题。
  2. 通过设置环境变量增加发现超时并减少发现流量:
    export FASTRTPS_DEFAULT_PROFILES_FILE=/path/to/fastdds_profile.xml
    
    在 XML 中设置:
    <discovery_config>
      <leaseDuration>PT60S</leaseDuration>
      <announcementPeriod>PT5S</announcementPeriod>
    </discovery_config>
    
    然后重启节点。

Dead Ends

Common approaches that don't work:

  1. Increasing the discovery timeout to 60 seconds via environment variable 70% fail

    Merely extending the timeout does not address the underlying scalability issue; discovery may still fail with more nodes.

  2. Restarting all nodes simultaneously 85% fail

    Simultaneous startup can cause a discovery storm, making the problem worse.