Rviz2 crashes with footprint drawing

[daisukes]

OS: Ubuntu focal
ROS2: foxy binary / foxy release src build (ros2/ros2@fc010c9#diff-215a2eb6c7ad8b20796a9fceb48f8cc7)
Issue

I have been working with Navigation2 and experiencing Rviz2 crash sometimes when the controller is ready and drawing (binary), sometimes the display option is changed (both src, binary).
It looks like footprint drawing causes this crash.

A workaround is to uncheck "Controller/Polygon", or use an src build and do not change the display option.

This could not reproduce 100% the crash but, got crash often.

1. ros2 launch nav2_bringup tb3_simulation_launch.py
2. give an initial position with "2D Pose Estimate"
3. uncheck/check "Controller" or "Controller/Polygon"

Backtrace with the binary build, crashed instantly (click to open)

#0  0x00007efcc7200649 in ?? () from /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0
#1  0x00007efc920f784e in Ogre::GLHardwareVertexBuffer::GLHardwareVertexBuffer(Ogre::HardwareBufferManagerBase*, unsigned long, unsigned long, Ogre::HardwareBuffer::Usage, bool) ()
   from /opt/ros/foxy/opt/rviz_ogre_vendor/lib/OGRE/RenderSystem_GL.so.1.12.1
#2  0x00007efc920ee907 in Ogre::GLHardwareBufferManager::createVertexBuffer(unsigned long, unsigned long, Ogre::HardwareBuffer::Usage, bool) ()
   from /opt/ros/foxy/opt/rviz_ogre_vendor/lib/OGRE/RenderSystem_GL.so.1.12.1
#3  0x00007efccb7be775 in Ogre::ManualObject::end() () from /opt/ros/foxy/opt/rviz_ogre_vendor/lib/libOgreMain.so.1.12.1
#4  0x00007efc91d0b21a in rviz_default_plugins::displays::PolygonDisplay::processMessage(std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const>) ()
   from /opt/ros/foxy/lib/librviz_default_plugins.so
#5  0x00007efc91b406f5 in rviz_common::MessageFilterDisplay<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > >::incomingMessage(std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const>) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#6  0x00007efc91b4096e in std::_Function_handler<void (std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const> const&), std::_Bind<void (rviz_common::MessageFilterDisplay<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > >::*(rviz_common::MessageFilterDisplay<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > >*, std::_Placeholder<1>))(std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const>)> >::_M_invoke(std::_Any_data const&, std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const> const&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#7  0x00007efc91b50994 in message_filters::CallbackHelper1T<std::shared_ptr<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const> const&, geometry_msgs::msg::PolygonStamped_<std::allocator<void> > >::call(message_filters::MessageEvent<geometry_msgs::msg::PolygonStamped_<std::allocator<void> > const> const&, bool) ()
   from /opt/ros/foxy/lib/librviz_default_plugins.so
#8  0x00007efc91b4ba6f in tf2_ros::MessageFilter<geometry_msgs::msg::PolygonStamped_<std::allocator<void> >, rviz_common::transformation::FrameTransformer>::transformReadyCallback(std::shared_future<geometry_msgs::msg::TransformStamped_<std::allocator<void> > > const&, unsigned long) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#9  0x00007efc915223f0 in ?? () from /opt/ros/foxy/lib/libtf2_ros.so
#10 0x00007efc914ce4b1 in tf2::BufferCore::testTransformableRequests() () from /opt/ros/foxy/lib/libtf2.so
#11 0x00007efc914cfd34 in tf2::BufferCore::setTransformImpl(tf2::Transform const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /opt/ros/foxy/lib/libtf2.so
#12 0x00007efc914d176d in tf2::BufferCore::setTransform(geometry_msgs::msg::TransformStamped_<std::allocator<void> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /opt/ros/foxy/lib/libtf2.so
#13 0x00007efc915287e6 in tf2_ros::TransformListener::subscription_callback(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, bool) ()
   from /opt/ros/foxy/lib/libtf2_ros.so
#14 0x00007efc91d65e9d in std::_Function_handler<void (std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >), std::_Bind<void (tf2_ros::TransformListener::*(tf2_ros::TransformListener*, std::_Placeholder<1>, bool))(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, bool)> >::_M_invoke(std::_Any_data const&, std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >&&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#15 0x00007efc91d6b6e1 in rclcpp::AnySubscriptionCallback<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void> >::dispatch(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, rclcpp::MessageInfo const&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#16 0x00007efc91d6bf4f in rclcpp::Subscription<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void>, rclcpp::message_memory_strategy::MessageMemoryStrategy<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void> > >::handle_message(std::shared_ptr<void>&, rclcpp::MessageInfo const&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#17 0x00007efccc19620c in ?? () from /opt/ros/foxy/lib/librclcpp.so
#18 0x00007efccc196acb in rclcpp::Executor::execute_subscription(std::shared_ptr<rclcpp::SubscriptionBase>) () from /opt/ros/foxy/lib/librclcpp.so
#19 0x00007efccc19728a in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () from /opt/ros/foxy/lib/librclcpp.so
#20 0x00007efccc19b94c in rclcpp::executors::SingleThreadedExecutor::spin() () from /opt/ros/foxy/lib/librclcpp.so
#21 0x00007efc91528a32 in ?? () from /opt/ros/foxy/lib/libtf2_ros.so
#22 0x00007efccbf9ecb4 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#23 0x00007efccaa3b609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#24 0x00007efccbddb103 in clone () from /usr/lib/x86_64-linux-gnu/libc.so.6

Backtrace with the src build (crashed after changing display option)

#0  0x00007f72174c5428 in tf2_ros::MessageFilter<geometry_msgs::msg::PolygonStamped_<std::allocator<void> >, rviz_common::transformation::FrameTransformer>::transformReadyCallback(std::shared_future<geometry_msgs::msg::TransformStamped_<std::allocator<void> > > const&, unsigned long) () from /opt/ros/foxy/lib/librviz_default_plugins.so
[Current thread is 1 (Thread 0x7f7216309700 (LWP 1292))]
(gdb) bt
#0  0x00007f72174c5428 in tf2_ros::MessageFilter<geometry_msgs::msg::PolygonStamped_<std::allocator<void> >, rviz_common::transformation::FrameTransformer>::transformReadyCallback(std::shared_future<geometry_msgs::msg::TransformStamped_<std::allocator<void> > > const&, unsigned long) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#1  0x00007f7216e9b430 in ?? () from /opt/ros/foxy/lib/libtf2_ros.so
#2  0x00007f724401c4b1 in tf2::BufferCore::testTransformableRequests() () from /opt/ros/foxy/lib/libtf2.so
#3  0x00007f724401dd34 in tf2::BufferCore::setTransformImpl(tf2::Transform const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /opt/ros/foxy/lib/libtf2.so
#4  0x00007f724401f76d in tf2::BufferCore::setTransform(geometry_msgs::msg::TransformStamped_<std::allocator<void> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /opt/ros/foxy/lib/libtf2.so
#5  0x00007f7216ea1826 in tf2_ros::TransformListener::subscription_callback(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, bool) () from /opt/ros/foxy/lib/libtf2_ros.so
#6  0x00007f72176dfe9d in std::_Function_handler<void (std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >), std::_Bind<void (tf2_ros::TransformListener::*(tf2_ros::TransformListener*, std::_Placeholder<1>, bool))(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, bool)> >::_M_invoke(std::_Any_data const&, std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >&&) ()
   from /opt/ros/foxy/lib/librviz_default_plugins.so
#7  0x00007f72176e56e1 in rclcpp::AnySubscriptionCallback<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void> >::dispatch(std::shared_ptr<tf2_msgs::msg::TFMessage_<std::allocator<void> > >, rclcpp::MessageInfo const&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#8  0x00007f72176e5f4f in rclcpp::Subscription<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void>, rclcpp::message_memory_strategy::MessageMemoryStrategy<tf2_msgs::msg::TFMessage_<std::allocator<void> >, std::allocator<void> > >::handle_message(std::shared_ptr<void>&, rclcpp::MessageInfo const&) () from /opt/ros/foxy/lib/librviz_default_plugins.so
#9  0x00007f724f2e828c in ?? () from /opt/ros/foxy/lib/librclcpp.so
#10 0x00007f724f2e8b4b in rclcpp::Executor::execute_subscription(std::shared_ptr<rclcpp::SubscriptionBase>) () from /opt/ros/foxy/lib/librclcpp.so
#11 0x00007f724f2e930a in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () from /opt/ros/foxy/lib/librclcpp.so
#12 0x00007f724f2edb0c in rclcpp::executors::SingleThreadedExecutor::spin() () from /opt/ros/foxy/lib/librclcpp.so
#13 0x00007f7216ea1a72 in ?? () from /opt/ros/foxy/lib/libtf2_ros.so
#14 0x00007f724f0f0cb4 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#15 0x00007f724db8d609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#16 0x00007f724ef2d103 in clone () from /usr/lib/x86_64-linux-gnu/libc.so.6

[jacobperron]

@daisukes I'm not able to reproduce the error. I've tried to follow the provided steps with Foxy Debian packages installed, but it doesn't look the the simulation is starting properly (the world looks empty with a laser scan visible). I see the following logs repeated in the terminal:

[rviz2-4] [INFO] [1597351344.603657719] [rviz2]: Waiting for the lifecycle_manager_navigation/is_active service...
[rviz2-4] [INFO] [1597351344.604034053] [rviz2]: Sending lifecycle_manager_navigation/is_active request
[controller_server-8] [INFO] [1597351345.049069669] [local_costmap.local_costmap]: Timed out waiting for transform from base_link to odom to become available, tf error: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[controller_server-8] [INFO] [1597351345.549273482] [local_costmap.local_costmap]: Timed out waiting for transform from base_link to odom to become available, tf error: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist

Is there a simpler example you can provide to reproduce the issue?

[daisukes]

@jacobperron Thank you for taking the time!

In order to enable simulation, you need to set Gazebo env before launching the simulation.

$ git clone https://github.com/ROBOTIS-GIT/turtlebot3_simulations.git
$ export GAZEBO_MODEL_PATH=`pwd`/turtlebot3_simulations/turtlebot3_gazebo/models/

[jacobperron]

@daisukes Thanks, I can see the world and robot model in Gazebo now, but I'm still getting the same TF-related logs about invalid "odom" frame. I don't see any footprint in RViz.

[jacobperron]

There was a recent fix to the tf2 message filter (ros2/geometry2#279). @daisukes Since the stack traces you provided indicate that the tf2 message filter is involved, could you try the latest Foxy release and see if you can still reproduce this?

[daisukes]

Thanks, I will check with the latest Foxy release.

but I'm still getting the same TF-related logs about invalid "odom" frame. I don't see any footprint in RViz.

Foxy has an issue with Fast-RTPS, which is recently fixed. Using cyclonedds may help the issue.

$ export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

[daisukes]

With the latest Foxy release (ros2/ros2@4109e26), it looks the issue has been gone.

So, I close this issue, and again thank you very much!