Core dump with gazebo 8.1 in gazebo_ros_pkgs

[osrf-migration]

Original report (archived issue) by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

Originally reported in the gazebo_ros_pkgs I was able to reproduce the problem easily updating gazebo8 from 8.0.0 to 8.1.0 (all changes from 8.0 to 8.1. Here is the useful part of the trace:

======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7efffed2e7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7efffed36e0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7efffed3a98c]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_assignERKS4_+0xaf)[0x7effff3b53ef]
/usr/lib/x86_64-linux-gnu/libgazebo_gui.so.8(+0x1adf2f)[0x7f0000957f2f]
/usr/lib/x86_64-linux-gnu/libgazebo_gui.so.8(_ZN6gazebo3gui12RenderWidget9AddPluginERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrIN3sdf7ElementEE+0xc6)[0x7f00009aa946]
/usr/lib/x86_64-linux-gnu/libgazebo_gui.so.8(_ZN6gazebo3gui12RenderWidget10AddPluginsERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS8_EE+0xf6)[0x7f00009aae66]
/usr/lib/x86_64-linux-gnu/libgazebo_gui.so.8(_ZN6gazebo3gui3runEiPPc+0x146)[0x7f000091c866]
gzclient[0x40275c]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7efffecd7830]
gzclient(_start+0x29)[0x4029c9]

It could be a problem related with the ABI although the checker is reporting no errors. I was suspecting on this change but I think it is ok.

My research guided me to this PR and what I can say is that problem is triggered in this call to gazebo::GUIPlugin::Create.

To reproduce the problem:

sudo aptitude install ros-kinetic-gazebo8-ros-pkgs
roslaunch gazebo_ros empty_world.launch

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

• Edited issue description

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

• Edited issue description

[osrf-migration]

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).

---

Yeah, there is a new virtual function added to GUIPlugin in pull request #2631, but it is at the end of the block, so I thought that would be ok

• https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/2631/backport-gui-plugin-fixes/diff#chg-gazebo/gui/GuiPlugin.hh

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

My tests have concluded that the culprit is the addition of an overload of the method AddPlugin which was not overloaded before and that kills the ABI.

I'm going to need the help of @iche033 to know the best way of fixing it.

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

After fighting with possible ABI breakages and get inconsistent results I've compiled the gazebo_ros_pkgs from source using gazebo 8.1.0 and ... the problems is still there.

So it is possible that we don't have an ABI problem but other memory related issue. Another easy way to trigger the problem is:

sudo apt-get install -y ros-kinetic-gazebo8-ros-pkgs
gazebo -g /opt/ros/kinetic/lib/libgazebo_ros_paths_plugin.so --verbose

Under gdb the stacktrace is:

#7  0x00007ffff7285edf in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign (
    __str="/home/jrivero/code/ros/ws/gazebo_ros_pkgs/devel/lib/libgazebo_ros_paths_plugin.so", this=<optimized out>) at /usr/include/c++/5/bits/basic_string.h:1095
#8  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator= (__str="/home/jrivero/code/ros/ws/gazebo_ros_pkgs/devel/lib/libgazebo_ros_paths_plugin.so", 
    this=<optimized out>) at /usr/include/c++/5/bits/basic_string.h:551
#9  gazebo::PluginT<gazebo::GUIPlugin>::Create (_filename=..., _name="/home/jrivero/code/ros/ws/gazebo_ros_pkgs/devel/lib/libgazebo_ros_paths_plugin.so")
    at /home/jrivero/code/gazebo/gazebo/common/Plugin.hh:199
#10 0x00007ffff72d88f6 in gazebo::gui::RenderWidget::AddPlugin_2 (this=this@entry=0x1b9cf90, _filename="/home/jrivero/code/ros/ws/gazebo_ros_pkgs/devel/lib/libgazebo_ros_paths_plugin.so", 
    _elem=std::shared_ptr (count 2, weak 1) 0x5026390) at /home/jrivero/code/gazebo/gazebo/gui/RenderWidget.cc:308
#11 0x00007ffff72d8e16 in gazebo::gui::RenderWidget::AddPlugins (this=0x1b9cf90, _pluginFilenames=std::vector of length 1, capacity 1 = {...})
    at /home/jrivero/code/gazebo/gazebo/gui/RenderWidget.cc:293
#12 0x00007ffff724a83b in gazebo::gui::run (_argc=<optimized out>, _argv=<optimized out>) at /home/jrivero/code/gazebo/gazebo/gui/GuiIface.cc:368
#13 0x000000000040275c in main (_argc=3, _argv=0x7fffffffd638) at /home/jrivero/code/gazebo/gazebo/gui/main.cc:29

I noticed that the libgazebo_ros_paths_plugin.so is defined and registered as a SystemPlugin and we are calling it via -g as it was a GUIPlugin. If I modify it to be a GUIPlugin then the crash is gone (although I get a black box in the gazebo GUI).

[osrf-migration]

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

---

One thing we could do is update GAZEBO_REGISTER_*_PLUGIN macros to add another symbol to the shared library: char *RegisteredPluginType. The gazebo plugin loading code would first call RegisterPlugin(). That method would set RegisteredPluginType = "model"; before it returned an instance of the plugin. The plugin loading code can check for that symbol, and if it exists change PluginT::type to that type. The code responsible for the -g option would need to double check the type and cast to the appropriate base class. For backwards compatibility the code handling -g should assume SystemPlugin to avoid the crash on libraries that have already been built.

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

That's is a good idea Shane. I'm a bit afraid of touching too much the plugin system in the middle of gazebo8 series. I think that the main problem is to leave the -g meaning loading a SystemPlugin (we agree on this). I can not see another solution that add a new parameter.

See Pull request #2716.

[osrf-migration]

Original comment by Jose Luis Rivero (Bitbucket: Jose Luis Rivero, GitHub: j-rivero).

---

• changed state from "new" to "resolved"

Pull request merged and released in gazebo 8.1.1