gzclient shutdown segmentation fault with ogre 1.10

[osrf-migration]

Original report (archived issue) by Tim Rakowski (Bitbucket: Tim Rakowski).

The original report had attachments: ogre-1.10.patch

---

This issue is related to https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/issues/2321/gazebo-800-startup-error-on-arch-linux. (#2321) I used both patches referenced by @iche033. Just to make sure, I attached the actual patch I used.

#!
Thread 1 "gzclient" received signal SIGSEGV, Segmentation fault.
0x00007ff0d62cd8ca in Ogre::SceneManagerEnumerator::shutdownAll() () from /usr/lib/libOgreMain.so.1.10.5
(gdb) bt
#0  0x00007ff0d62cd8ca in Ogre::SceneManagerEnumerator::shutdownAll() () at /usr/lib/libOgreMain.so.1.10.5
#1  0x00007ff0d62a0ea7 in Ogre::Root::shutdown() () at /usr/lib/libOgreMain.so.1.10.5
#2  0x00007ff0d62a281c in Ogre::Root::~Root() () at /usr/lib/libOgreMain.so.1.10.5
#3  0x00007ff0ddb0de2f in gazebo::rendering::RenderEngine::Fini() (this=this@entry=0x7ff0df44d240 <SingletonT<gazebo::rendering::RenderEngine>::GetInstance()::t>) at /home/racko/aur/gazebo/src/gazebo-8.0.0/gazebo/rendering/RenderEngine.cc:368
#4  0x00007ff0ddb16383 in gazebo::rendering::fini() () at /home/racko/aur/gazebo/src/gazebo-8.0.0/gazebo/rendering/RenderingIface.cc:66
#5  0x00007ff0de8b69cd in gazebo::gui::fini() () at /home/racko/aur/gazebo/src/gazebo-8.0.0/gazebo/gui/GuiIface.cc:215
#6  0x00007ff0de8b8110 in gazebo::gui::run(int, char**) (_argc=<optimized out>, _argv=0x7fffffffda48)
    at /home/racko/aur/gazebo/src/gazebo-8.0.0/gazebo/gui/GuiIface.cc:370
#7  0x000000010000376c in main(int, char**) (_argc=2, _argv=0x7fffffffda48) at /home/racko/aur/gazebo/src/gazebo-8.0.0/gazebo/gui/main.cc:29

Here is the gazebo code causing the seg fault:

https://github.com/osrf/gazebo/blob/06333d5dbdf94f2e09eae8f8a6f8e2bacb3f0455/gazebo/rendering/RenderEngine.cc#L354

#!c++

    this->dataPtr->root->shutdown();

    // ...

    try 
    {   
      delete this->dataPtr->root; // SIGSEGV
    }   
    catch(...)
    {   
    }   

Calling Ogre::Root::~Root after calling Ogre::Root::shutdown seems to fail. Which is not documented, so either Ogre should fix it or it may be caused by something else gazebo does.

I think that the this-pointer in the SceneManagerEnumerator::shutdown call (called from Root::~Root) is 0, but I honestly was not able to figure out how this can be caused by "Root::~Root followed by Root::shutdown":

#!

Dump of assembler code for function _ZN4Ogre22SceneManagerEnumerator11shutdownAllEv:
   0x00007ff0d62cd8c0 <+0>:	push   %rbp
   0x00007ff0d62cd8c1 <+1>:	lea    0x20(%rdi),%rbp
   0x00007ff0d62cd8c5 <+5>:	push   %rbx
   0x00007ff0d62cd8c6 <+6>:	sub    $0x8,%rsp
=> 0x00007ff0d62cd8ca <+10>:	mov    0x30(%rdi),%rbx
...
(gdb) p $edi
$1 = 0

Possible Solution

According to https://bitbucket.org/sinbad/ogre/src/695d30368b416df2f90d570f747e4c656e0462f3/OgreMain/src/OgreRoot.cpp?at=v1-8&fileviewer=file-view-default#OgreRoot.cpp-288,
Ogre::Root::~Root calls Ogre::Root::shutdown, so you might as well remove the shutdown call, which fixes the issue in my tests.

Edit

In hindsight I did the obvious thing and created the smallest test possible to show that Root::shutdown followed by Root::~Root leads to seg fault:

#!c++

#include <OgreRoot.h>

int main()
{
    Ogre::Root root;
    root.shutdown();
    return 0;
}

#!
$ g++ -I/usr/include/OGRE -lOgreMain -lboost_system main.cpp
$ gdb ./a.out
...
*-*-* OGRE Initialising
*-*-* Version 1.10.5 (Xalafu)
*-*-* OGRE Shutdown

Program received signal SIGSEGV, Segmentation fault.
0x00007f5b574988ca in Ogre::SceneManagerEnumerator::shutdownAll() () from /usr/lib/libOgreMain.so.1.10.5
(gdb) bt
#0  0x00007f5b574988ca in Ogre::SceneManagerEnumerator::shutdownAll() () from /usr/lib/libOgreMain.so.1.10.5
#1  0x00007f5b5746bea7 in Ogre::Root::shutdown() () from /usr/lib/libOgreMain.so.1.10.5
#2  0x00007f5b5746d81c in Ogre::Root::~Root() () from /usr/lib/libOgreMain.so.1.10.5
#3  0x00000001000016aa in main ()

Edit 2

I was able to find the (actually quite obvious) root cause:

https://bitbucket.org/sinbad/ogre/src/b42cf1abf81571baaefd7c6b92ebfd701896686f/OgreMain/src/OgreRoot.cpp?at=v1-10-5&fileviewer=file-view-default#OgreRoot.cpp-1030

#!c++

    void Root::shutdown(void)
    {
        if(mActiveRenderer)
            mActiveRenderer->_setViewport(NULL);

        // Since background thread might be access resources,
        // ensure shutdown before destroying resource manager.
        mResourceBackgroundQueue->shutdown();
        mWorkQueue->shutdown();

        SceneManagerEnumerator::getSingleton().shutdownAll(); // uses mSceneManagerEnum
        shutdownPlugins();
        OGRE_DELETE mSceneManagerEnum; // <==== deletes mSceneManagerEnum

        OGRE_DELETE mShadowTextureManager;
        ShadowVolumeExtrudeProgram::shutdown();
        ResourceGroupManager::getSingleton().shutdownAll();

        // Destroy pools
        ConvexBody::_destroyPool();


        mIsInitialised = false;

        LogManager::getSingleton().logMessage("*-*-* OGRE Shutdown");
    }

So calling Root::shutdown once deletes mSceneManagerEnum, and calling it a second time will access it again. Generally, Root::shutdown must not be called twice to prevent double deletes ... But this really looks like a Ogre issue since it isn't documented.

I created a ticket for this in the Ogre JIRA: https://ogre3d.atlassian.net/browse/OGRE-576

[osrf-migration]

Original comment by Tim Rakowski (Bitbucket: Tim Rakowski).

---

• Edited issue description

[osrf-migration]

Original comment by Tim Rakowski (Bitbucket: Tim Rakowski).

---

• Edited issue description

[osrf-migration]

Original comment by Tim Rakowski (Bitbucket: Tim Rakowski).

---

• Edited issue description

[osrf-migration]

Original comment by Ian Chen (Bitbucket: Ian Chen, GitHub: iche033).

---

thanks for looking into this. I think it's safe to remove the Ogre::Root::shutdown call in gazebo and let ogre take care of it in its destructor. Would you like to create a pull request for this targeting the gazebo8 branch?

[osrf-migration]

Original comment by Tim Rakowski (Bitbucket: Tim Rakowski).

---

Done: https://osrf-migration.github.io/gazebo-gh-pages/#!/osrf/gazebo/pull-requests/2761

[osrf-migration]

Original comment by Ian Chen (Bitbucket: Ian Chen, GitHub: iche033).

---

• changed state from "new" to "resolved"

Fixed issue #2324

Ogre::Root::shutdown followed by Ogre::Root::~Root leads to a
segmentation fault. Since ~Root calls shutdown anyhow, the shutdown call
in RenderEngine::Fini can be removed.

→ <<cset 27992b0>>

[osrf-migration]

Original comment by Ian Chen (Bitbucket: Ian Chen, GitHub: iche033).

---

Merged in tim_rakowski/gazebo/gazebo8_ogre_1.10_shutdown_segfault_fix (pull request #2761)

Fixed issue #2324

Approved-by: Ian Chen ichen@osrfoundation.org
Approved-by: Louise Poubel lupoubel@hotmail.com

→ <<cset ef73f34>>