dynamic_bridge problem with unstable listener

[ArkadiuszNiemiec]

Bug report

Required Info:

• Operating System:
  • Ubuntu 16.04
• Installation type:
  • source
• Version or commit hash:
  • current master
• DDS implementation:
  • Fast-RTPS

• Client library (if applicable):

Steps to reproduce issue

I am able to reproduce it using example from README. Start dynamic_bridge, run ROS1 talker and ROS2 listener. Then CTRL+C on listener and run it again. After few such cycles listener no longer prints any message. If you look at the ros2 topic echo you can see that there's nothing on the ROS2 topic.

This is what bridge outputs:

xxx@xxx:~/ros2_ws$ ros2 run ros1_bridge dynamic_bridge
created 1to2 bridge for topic '/chatter' with ROS 1 type 'std_msgs/String' and ROS 2 type 'std_msgs/String'
[INFO] [ros1_bridge]: Passing message from ROS 1 std_msgs/String to ROS 2 std_msgs/String (showing msg only once per type)
removed 1to2 bridge for topic '/chatter'
created 1to2 bridge for topic '/chatter' with ROS 1 type 'std_msgs/String' and ROS 2 type 'std_msgs/String'

Expected behavior

Bridge should be able to recover from network problems

Actual behavior

Bridge stops passing messages from ROS1 to ROS2

[ArkadiuszNiemiec]

I am trying to debug it by myself. I have changed RCUTILS_LOG_INFO_ONCE_NAMED to RCUTILS_LOG_INFO_NAMED in factory.hpp and now it prints it every message even if ROS2 topic is empty.

[dhood]

What I have understood is that you are creating and destroying a ros2 listener, and expecting the 1to2 bridge to get created and destroyed with it. That is reasonable to expect, if it's not working it sounds like a bug.

It's not clear to me what behaviour you're seeing when things stop working correctly. Is the issue that the 1to2 bridge does not get created when the listener is restarted? Or does the bridge get created but the listener does not receive anything?

now it prints it every message even if ROS2 topic is empty.

Can you clarify what you mean here?

[ArkadiuszNiemiec]

I think that the bridge is created: you can see that new ROS2 topic exists but the listener does not receive anything. Additionally if you try to use ros2 topic echo on that topic it does not print anything.

[dhood]

I can confirm this behaviour is a bug. The trigger seems to be when the bridge itself is re-created multiple times.

I tested with opensplice too and saw it wasn't an issue, so it's likely that this is specific to using fastrtps.

Sometimes fastrtps can have issues with bursty data but even with these settings that have been known to address that in some cases uncommented, it was still an issue.

Enabling debug logging in rcl (adding rcutils_logging_set_logger_level("rcl", RCUTILS_LOG_SEVERITY_DEBUG); to the dynamic_bridge)/looking at wireshark it seems that the bridge is publishing as expected, the listeners just aren't connecting.

If you or anyone else has a chance to investigate this further it'd be good to know what's causing this.

Things that would help are to enable debug logging in fastrtps (see ros2/rmw_fastrtps#156) or to add info on the number of matched subscribers for rmw_fastrtps as mentioned in ros2/ros2#380 and ros2/rmw#120. There might be some settings in fastrtps that can be tweaked in order to get the listeners to connect more reliably.

[dirk-thomas]

I would recommend to come up with a reproducible example which doesn't involve the bridge since it sounds to me that it is a pure FastRTPS problem. Then we could more easily share the example with the maintainers of FastRTPS / fill a ticket upstream for them to look into the problem and hopefully address it.

[ArkadiuszNiemiec]

Missclicked, sorry. Can we assume that it's fixed by the FastRTPS devs?

[dirk-thomas]

Can we assume that it's fixed by the FastRTPS devs?

Please try to reproduce the problem with the latest state and comment if the problem still exists or has been resolved.

[ArkadiuszNiemiec]

I think it's now working as intended on current Ubuntu repo version. What I did:

terminal 1:

source /opt/ros/melodic/setup.bash 
roscore

terminal 2:

source /opt/ros/melodic/setup.bash 
rosrun rospy_tutorials talker

terminal 3:

source /opt/ros/melodic/setup.bash
source /opt/ros/bouncy/local_setup.bash
ros2 run ros1_bridge dynamic_bridge

terminal 4:

source /opt/ros/melodic/setup.bash
source /opt/ros/bouncy/local_setup.bash
ros2 run examples_rclpy_minimal_subscriber subscriber_old_school topic:=chatter

I was killing the subscriber and there was no problem at all. I will close it and thank you all.