proto used by executable file and shared lib crash

[E-HERO]

I met a problem that my executable file used a static lib named A which contain a proto and my shared lib(.so file) also use the same static lib A. When my executable file load (called by dlopen) the shared lib，the program crash (core segment).similar with https://github.com/google/protobuf/issues/1489
But I can't replace the static lib A with shared lib (A.so), because the lib is too complicated and I have no right to change it.
I have tried another way that I remove static lib A from the shared lib, therefore these symbols in static lib A is undefined in the shared lib (nm can prove it). I think the symbols in static lib A have already been in memory when my executable file startup, so when executable file dlopen the shared lib it will search static lib A's symbols in memory. However, when dlopen the shared lib, it dlopen fail because of undefined symbol(not core segment error).

So, I am very appreciated for any suggestion to solve the problem.

the core information ：
#0 google::protobuf::internal::Mutex::Lock (this=0x0) at google/protobuf/stubs/common.cc:302
#1 0x00007f9838e1f9b0 in google::protobuf::internal::MutexLock::MutexLock (this=, mu=0x0) at ./google/protobuf/stubs/common.h:1866
#2 0x00007f9838e1f9b0 in google::protobuf::internal::LogMessage::Finish (this=0x7fff93979090) from ../plugin/libzacdefaultworker.so
#3 0x00007f9838e71b1f in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::AddFile (this=0x285d018, file=..., value=...)
at ./google/protobuf/descriptor.pb.h:2682
#4 0x00007f9838e6cc30 in google::protobuf::EncodedDescriptorDatabase::Add (this=0x285d010, encoded_file_descriptor=0x7f98390d9168, size=960)
at /usr/lib/gcc/x86_64-redhat-linux/3.4.5/../../../../include/c++/3.4.5/bits/stl_pair.h:144
#5 0x00007f9838e2de11 in google::protobuf::DescriptorPool::InternalAddGeneratedFile (encoded_file_descriptor=0x7f98390d9168, size=960) at google/protobuf/descriptor.cc:862
#6 0x00007f9838eca0fa in protobuf_AddDesc_service_2eproto () at msg/service.pb.cc:260
#7 0x00007f9838ed8483 in StaticDescriptorInitializer_service_2eproto::StaticDescriptorInitializer_service_2eproto (this=0x1ac8938 <static_descriptor_initializer_service_2eproto_>)
at msg/service.pb.cc:283
#8 0x00007f9838ed65fd in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at msg/service.pb.cc:285
#9 0x00007f9838ed6612 in _GLOBAL__sub_I_service.pb.cc(void) () at msg/service.pb.cc:3205
#10 0x00007f9838ed97b6 in __do_global_ctors_aux () from ../plugin/libzacdefaultworker.so
#11 0x00007f9838a7fd37 in _init () from ../plugin/libzacdefaultworker.so
#12 0x00007f98389ec0a0 in ?? () from ../plugin/libzacdefaultworker.so
#13 0x00007f985368f7cd in call_init.part () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#14 0x00007f985368f8f3 in _dl_init_internal () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#15 0x00007f9853693a28 in dl_open_worker () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#16 0x00007f985368f694 in _dl_catch_error () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#17 0x00007f985369321b in _dl_open () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#18 0x00007f9852a0601b in dlopen_doit () from /opt/compiler/gcc-4.8.2/lib/libdl.so.2
#19 0x00007f985368f694 in _dl_catch_error () from /opt/compiler/gcc-4.8.2/lib64/ld-linux-x86-64.so.2
#20 0x00007f9852a065cd in _dlerror_run () from /opt/compiler/gcc-4.8.2/lib/libdl.so.2
#21 0x00007f9852a060b1 in dlopen@@GLIBC_2.2.5 () from /opt/compiler/gcc-4.8.2/lib/libdl.so.2

[acozzette]

Have you tried bundling static lib A into your .so file and having your main executable not link in A? I think ultimately the one-definition rule (ODR) applies and so you have to make sure the symbols for your proto are only visible in one place. Doing dynamic loading with protos can get tricky.

[E-HERO]

Thank you for your replying. It doesn't work, I have tried. Some codes in main executable are dependent with static lib A. There will be a compile error "undefined reference" if I remove A from my main executable.

[goodbuu]

Your protobuf doesn't look like original version. Maybe this crash was caused by your modified code.

[E-HERO]

Thanks for all replying, I solved the problem. The crash is caused by proto initialized more than once(I guess, just like this mentioned). The useful way is that I applied the right for changing A lib and build a shared lib A.so, therefore my bin and my so file(not A.so) linking to lib A in dynamic way. Because of A.so will only loaded for once, it can't initialize the proto contained in A lib twice. So I solved it.

The second thing I found is that so file won't search symbols contained in bin file which I tested myself and asked for advice from a experienced man. So if you have the same idea that so files search symbols contained in bin, maybe it doesn't work.

The last thing I found is someone in my company met the similar problem, they use dlopen with DEEPBIND which seems work(but this brought some other problems because of other things so they found me asking for advice). If my theory is right that the crash is caused by proto initialized more than once, this way is understandable. DEEPBIND parameter will use a separate scope from bin, so the proto is initialized twice but they are in different scopes and works.

Thanks everyone helped me in the process, thank you very much.