ROS1 bridge failures on nightly packaging jobs

[clalancette]

The nightly linux packaging job had failures in the ROS 1 bridge: https://ci.ros2.org/view/packaging/job/packaging_linux/1457/testReport/junit/(root)/projectroot/test_topics_across_dynamic_bridge__rmw_connext_cpp/ (there are quite a few others as well). The tests seem to be timing out the tests, with some of the output being:

launch_testing.util.proc_lookup.NoMatchingProcessException: No data recorded for proc ros1_talker_ros2_listener_across_dynamic_bridge__listener-4 

[dirk-thomas]

While the merge PR reduces the number of test failures there are still a few left.

[clalancette]

Here's the link the latest failures: https://ci.ros2.org/view/packaging/job/packaging_linux/1460/#showFailuresLink

[dirk-thomas]

I see the exact same test failures locally (using a default build without any customizations).

[clalancette]

Still seeing this problem last night: https://ci.ros2.org/view/packaging/job/packaging_linux/1461/

[hidmic]

Resolved by ros2/ros1_bridge#194. @dirk-thomas or @clalancette, mind to close the issue? (I don't have the power)

[clalancette]

Unfortunately, this came back last night: https://ci.ros2.org/view/packaging/job/packaging_linux/1468/testReport/junit/(root)/projectroot/test_topics_across_dynamic_bridge__rmw_connext_cpp/. @hidmic mind taking a look?

[dirk-thomas]

For the record: the melt failing tests are completely different than before. So this is now a different problem.

[hidmic]

Yeap, that's new. I saw some of them popped up yesterday already while I was running CI for some launch testing PRs. My gut feeling is that it has something to do with ros2/ros1_bridge@43e2fca. I'll take a look.

[hidmic]

Alright folks, this isn't new but just another failure mode of the well known issue that Fast-RTPS has with publishers and/or subscribers that use different typesupports (i.e. for different languages) for the same topic within the same process.

Our dynamic_bridge being a ROS2 node publishes to the /rosout topic via a C typesupport publisher within rcl and then attempts to bridge it to ROS1, subscribing to the same topic via a C++ typesupport subscriber. We had not seen an instance of this because before ros2/ros1_bridge@43e2fca we were not capable of bridging /rosout from ROS2 to ROS1. Of course, none of this happens with e.g. Opensplice.

Fast-RTPS devs said a fix was somewhat tricky, so I either we apply the known workaround or we disable bridge tests for Fast-RTPS. Or help/submit a fix upstream ourselves.

[dirk-thomas]

we disable bridge tests for Fast-RTPS

I don't think this is a viable option since a) we loose coverage for the default RMW impl. and b) users using the bridge would still run into this problem.

While we could look into working on a fix for upstream I think we need a workaround at least for the near term.

[hidmic]

Let's go with the workaround then. I'll leave a TODO for future correction.