Segfault in test_rdf_integration

[henningkayser]

Description

When I run colcon test, test_rdf_integration crashes with a segfault for me.

Your environment

• ROS Distro: Rolling
• OS Version: Ubuntu 22.04
• Source build: main

Steps to reproduce

Run colcon test

Expected behaviour

Happy tests

Actual behaviour

Unhappy test_rdf_integration with segfault below

Backtrace or Console output

build/moveit_ros_planning/Testing/20220331-1631/Test.xml: 6 tests, 0 errors, 1 failure, 0 skipped
- rdf_loader_test_launch_test_rdf_integration.test.py
  <<< failure message
    -- run_test.py: invoking following command in '/home/ubuntu/ws_ros2/build/moveit_ros_planning/rdf_loader':
     - ros2 test /home/ubuntu/ws_ros2/src/moveit2/moveit_ros/planning/rdf_loader/test/launch/test_rdf_integration.test.py --junit-xml=/home/ubuntu/ws_ros2/build/moveit_ros_planning/test_results/moveit_ros_planning/rdf_loader_test_launch_test_rdf_integration.test.py.xunit.xml --package-name=moveit_ros_planning
    Running with ROS_DOMAIN_ID 1
    ROS_DOMAIN_ID 1
    [INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2022-03-31-16-31-04-114273-moveit2-rolling-102947
    [INFO] [launch]: Default logging verbosity is set to INFO
    test_gtest_run_complete (moveit_ros_planning.TestGTestWaitForCompletion) ... [INFO] [test_rdf_integration-1]: process started with pid [102949]
    [INFO] [boring_string_publisher-2]: process started with pid [102951]
    [INFO] [boring_string_publisher-3]: process started with pid [102953]
    [test_rdf_integration-1] [INFO] [1648744264.210885226] [moveit_rdf_loader.rdf_loader]: Loaded robot model in 0.00180847 seconds
    [test_rdf_integration-1] [ERROR] [1648744274.221197691] [non_existent]: Could not find parameter does_not_exist and did not receive does_not_exist via std_msgs::msg::String subscription within 10.000000 seconds.
    [test_rdf_integration-1] Error:   Error document empty.
    [test_rdf_integration-1]          at line 100 in ./urdf_parser/src/model.cpp
    [test_rdf_integration-1] Failed to parse robot description using: urdf_xml_parser/URDFXMLParser
    [test_rdf_integration-1] [INFO] [1648744284.229708675] [moveit_rdf_loader.rdf_loader]: Unable to parse URDF
    [test_rdf_integration-1] [INFO] [1648744284.271172365] [moveit_rdf_loader.rdf_loader]: Loaded robot model in 0.0311817 seconds
    [test_rdf_integration-1] [INFO] [1648744284.448646240] [moveit_rdf_loader.rdf_loader]: Loaded robot model in 0.169545 seconds
    [test_rdf_integration-1] cannot publish data, at ./src/rmw_publish.cpp:59 during '__function__'
    [test_rdf_integration-1] Fail in delete datareader, at ./src/rmw_service.cpp:103 during '__function__'
    [test_rdf_integration-1] [ERROR] [1648744284.588085308] [executor.rclcpp]: Error in destruction of rcl service handle: Fail in delete datawriter, at ./src/rmw_service.cpp:117, at ./src/rcl/service.c:198
    [test_rdf_integration-1] Stack trace (most recent call last) in thread 103009:
    [test_rdf_integration-1] #18   Object "", at 0xffffffffffffffff, in 
    [test_rdf_integration-1] #17   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f19412609ff, in 
    [test_rdf_integration-1] #16   Object "/usr/lib/x86_64-linux-gnu/libc.so.6", at 0x7f19411ceb42, in 
    [test_rdf_integration-1] #15   Object "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30", at 0x7f19414602cb, in 
    [test_rdf_integration-1] #14   Object "/home/ubuntu/ws_ros2/install/moveit_ros_planning/lib/moveit_ros_planning/test_rdf_integration", at 0x56372f2d58df, in std::thread::_State_impl<std::thread::_Invoker<std::tuple<RDFIntegration_executor_Test::TestBody()::$_0> > >::~_State_impl()
    [test_rdf_integration-1] #13   Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416a1b2a, in rclcpp::Node::~Node()
    [test_rdf_integration-1] #12   Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f194167bdf9, in 
    [test_rdf_integration-1] #11   Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416a2f39, in 
    [test_rdf_integration-1] #10   Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416aa669, in rclcpp::node_interfaces::NodeParameters::~NodeParameters()
    [test_rdf_integration-1] #9    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f194167bdf9, in 
    [test_rdf_integration-1] #8    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416bb591, in 
    [test_rdf_integration-1] #7    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f194167bdf9, in 
    [test_rdf_integration-1] #6    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416f2060, in rclcpp::ServiceBase::~ServiceBase()
    [test_rdf_integration-1] #5    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f194167bdf9, in 
    [test_rdf_integration-1] #4    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416e8224, in 
    [test_rdf_integration-1] #3    Object "/opt/ros/rolling/lib/librclcpp.so", at 0x7f19416e4158, in 
    [test_rdf_integration-1] #2    Object "/opt/ros/rolling/lib/librcl.so", at 0x7f193fdd95bd, in 
    [test_rdf_integration-1] #1    Object "/opt/ros/rolling/lib/librmw_fastrtps_shared_cpp.so", at 0x7f193a883096, in rmw_fastrtps_shared_cpp::__rmw_destroy_service(char const*, rmw_node_s*, rmw_service_s*)
    [test_rdf_integration-1] #0    Object "/opt/ros/rolling/lib/librmw_fastrtps_shared_cpp.so", at 0x7f193a86fb84, in rmw_fastrtps_shared_cpp::create_rmw_gid(char const*, eprosima::fastrtps::rtps::GUID_t const&)
    [test_rdf_integration-1] Segmentation fault (Address not mapped to object [0x5600000031c0])
    [ERROR] [test_rdf_integration-1]: process has died [pid 102949, exit code -11, cmd '/home/ubuntu/ws_ros2/install/moveit_ros_planning/lib/moveit_ros_planning/test_rdf_integration --ros-args --params-file /tmp/launch_params_kpktgyo9'].
    ok
    
    ----------------------------------------------------------------------
    Ran 1 test in 20.549s
    
    OK
    [INFO] [boring_string_publisher-3]: sending signal 'SIGINT' to process[boring_string_publisher-3]
    [INFO] [boring_string_publisher-2]: sending signal 'SIGINT' to process[boring_string_publisher-2]
    [boring_string_publisher-3] [INFO] [1648744284.696220313] [rclcpp]: signal_handler(signum=2)
    [boring_string_publisher-2] [INFO] [1648744284.696695557] [rclcpp]: signal_handler(signum=2)
    [INFO] [boring_string_publisher-3]: process has finished cleanly [pid 102953]
    [INFO] [boring_string_publisher-2]: process has finished cleanly [pid 102951]
    test_gtest_pass (moveit_ros_planning.TestGTestProcessPostShutdown) ... FAIL
    
    ======================================================================
    FAIL: test_gtest_pass (moveit_ros_planning.TestGTestProcessPostShutdown)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/ubuntu/ws_ros2/src/moveit2/moveit_ros/planning/rdf_loader/test/launch/test_rdf_integration.test.py", line 78, in test_gtest_pass
        launch_testing.asserts.assertExitCodes(proc_info, process=integration_node)
      File "/opt/ros/rolling/lib/python3.10/site-packages/launch_testing/asserts/assert_exit_codes.py", line 62, in assertExitCodes
        assert info.returncode in allowable_exit_codes, 'Proc {} exited with code {}'.format(
    AssertionError: Proc test_rdf_integration-1 exited with code -11
    
    ----------------------------------------------------------------------
    Ran 1 test in 0.001s
    
    FAILED (failures=1)
    -- run_test.py: return code 1
    -- run_test.py: verify result file '/home/ubuntu/ws_ros2/build/moveit_ros_planning/test_results/moveit_ros_planning/rdf_loader_test_launch_test_rdf_integration.test.py.xunit.xml'
  >>>
build/moveit_ros_planning/test_results/moveit_ros_planning/rdf_loader_test_launch_test_rdf_integration.test.py.xunit.xml: 2 tests, 0 errors, 1 failure, 0 skipped
- moveit_ros_planning.TestGTestProcessPostShutdown test_gtest_pass
  <<< failure message
    Traceback (most recent call last):
      File "/home/ubuntu/ws_ros2/src/moveit2/moveit_ros/planning/rdf_loader/test/launch/test_rdf_integration.test.py", line 78, in test_gtest_pass
        launch_testing.asserts.assertExitCodes(proc_info, process=integration_node)
      File "/opt/ros/rolling/lib/python3.10/site-packages/launch_testing/asserts/assert_exit_codes.py", line 62, in assertExitCodes
        assert info.returncode in allowable_exit_codes, 'Proc {} exited with code {}'.format(
    AssertionError: Proc test_rdf_integration-1 exited with code -11
  >>>

Summary: 781 tests, 0 errors, 2 failures, 3 skipped

[vatanaksoytezer]

Wonder why CI didn't catch this... Might it be related to this: 6fa02b7?

[DLu]

I have isolated the problem to TEST(RDFIntegration, executor) which was added by @corycrean in #1050.

I'm not quite wrapping my head around the thread issue at the moment, but somehow when this test is being completed, the destructors are called in the wrong order, or some such.

[DLu]

[DLu]

(For cross referencing, this test is being temporarily disabled in #1170 )

[vatanaksoytezer]

Eventually fixed within #1170. Kudos to @JafarAbdi, we can close this!