static initializers (StaticDescriptorInitializer_*_2eproto) are MemorySanitizer-unclean

[tamird]

It seems that any executable which links libprotoc is msan-unclean:

$ cat test.cc
int main() { }

$ clang++ -fsanitize=memory -fsanitize-memory-track-origins -fno-omit-frame-pointer test.cc libprotoc.a && ./a.out
==386==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0xd4c37a in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::ValidateSymbolName(std::string const&) /go/src/github.com/cockroachdb/c-protobuf/descriptor_database.cc:238:24
    #1 0xd4000e in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::AddSymbol(std::string const&, std::pair<void const*, int>) /go/src/github.com/cockroachdb/c-protobuf/descriptor_database.cc:93:8
    #2 0xcec941 in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::AddFile(google::protobuf::FileDescriptorProto const&, std::pair<void const*, int>) /go/src/github.com/cockroachdb/c-protobuf/descriptor_database.cc:68:10
    #3 0xcd7ca6 in google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) /go/src/github.com/cockroachdb/c-protobuf/descriptor_database.cc:312:12
    #4 0x65247f in google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) /go/src/github.com/cockroachdb/c-protobuf/descriptor.cc:1128:3
    #5 0x4965f2 in google::protobuf::protobuf_AddDesc_google_2fprotobuf_2fany_2eproto() /go/src/github.com/cockroachdb/c-protobuf/any.pb.cc:82:3
    #6 0x4a41f9 in google::protobuf::StaticDescriptorInitializer_google_2fprotobuf_2fany_2eproto::StaticDescriptorInitializer_google_2fprotobuf_2fany_2eproto() /go/src/github.com/cockroachdb/c-protobuf/any.pb.cc:98:5
    #7 0x41f601 in __cxx_global_var_init /go/src/github.com/cockroachdb/c-protobuf/any.pb.cc:100:3
    #8 0x41f618 in _GLOBAL__sub_I_any.pb.cc /go/src/github.com/cockroachdb/c-protobuf/any.pb.cc
    #9 0x1a08b8c in __libc_csu_init (/go/src/github.com/cockroachdb/cockroach/proto_test/a.out+0x1a08b8c)
    #10 0x7faf62eacad4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ad4)
    #11 0x4200c3 in _start (/go/src/github.com/cockroachdb/cockroach/proto_test/a.out+0x4200c3)

  Uninitialized value was created by a heap allocation
    #0 0x48df92 in operator new(unsigned long) (/go/src/github.com/cockroachdb/cockroach/proto_test/a.out+0x48df92)
    #1 0x7faf63e32e98 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbee98)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /go/src/github.com/cockroachdb/c-protobuf/descriptor_database.cc:238:24 in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::ValidateSymbolName(std::string const&)
Exiting

This reproduces with libprotoc v3.0.0-beta-2 and the following linux and clang versions:

$ uname -a
Linux 3a9d0009bc6f 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015 x86_64 GNU/Linux
$ clang --version
clang version 3.8.0 (trunk 255169)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /third_party/llvm-build/Release+Asserts/bin

[tamird]

This was previously reported in #912.

[tamird]

Simpler repro:

curl --location https://github.com/google/protobuf/releases/download/v3.0.0-beta-2/protobuf-cpp-3.0.0-beta-2.tar.gz | tar zx
cd protobuf-3.0.0-beta-2
./configure CPPFLAGS='-fsanitize=memory -fsanitize-memory-track-origins' LDFLAGS='-fsanitize=memory'
make check -j4
...
==11781==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7f23fcce1ec2 in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::ValidateSymbolName(std::string const&) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x9feec2)
    #1 0x7f23fccd6ef6 in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::AddSymbol(std::string const&, std::pair<void const*, int>) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x9f3ef6)
    #2 0x7f23fcc8c0a3 in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::AddFile(google::protobuf::FileDescriptorProto const&, std::pair<void const*, int>) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x9a90a3)
    #3 0x7f23fcc76a9e in google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x993a9e)
    #4 0x7f23fc8ecdc5 in google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x609dc5)
    #5 0x7f23fc851e02 in google::protobuf::protobuf_AddDesc_google_2fprotobuf_2fany_2eproto() (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x56ee02)
    #6 0x7f23fc85f873 in google::protobuf::StaticDescriptorInitializer_google_2fprotobuf_2fany_2eproto::StaticDescriptorInitializer_google_2fprotobuf_2fany_2eproto() (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x57c873)
    #7 0x7f23fc54ba8e in __cxx_global_var_init (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x268a8e)
    #8 0x7f23fc54baa8 in _GLOBAL__sub_I_any.pb.cc (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x268aa8)
    #9 0x7f23fe51f9f9  (/lib64/ld-linux-x86-64.so.2+0xe9f9)
    #10 0x7f23fe51fae2  (/lib64/ld-linux-x86-64.so.2+0xeae2)
    #11 0x7f23fe5121c9  (/lib64/ld-linux-x86-64.so.2+0x11c9)

  Uninitialized value was created by a heap allocation
    #0 0x48b2c2 in operator new(unsigned long) (/protobuf/protobuf-3.0.0-beta-2/src/.libs/lt-protoc+0x48b2c2)
    #1 0x7f23fac1ae98 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbee98)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/protobuf/protobuf-3.0.0-beta-2/src/.libs/libprotobuf.so.10+0x9feec2) in google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::pair<void const*, int> >::ValidateSymbolName(std::string const&)
Exiting
Makefile:6623: recipe for target 'unittest_proto_middleman' failed
make[1]: *** [unittest_proto_middleman] Error 77
make[1]: Leaving directory '/protobuf/protobuf-3.0.0-beta-2/src'
Makefile:1187: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

[tamird]

On master (7619505), this fails with something even more innocuous-looking:

$ make check -j4
==15230==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7f36a4973821 in std::string::_Rep::_M_dispose(std::allocator<char> const&) /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/basic_string.h:240:8
    #1 0x7f36a4973821 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/basic_string.h:547
    #2 0x7f36a4973821 in google::protobuf::util::Status::Status(google::protobuf::util::error::Code, google::protobuf::StringPiece) /go/src/github.com/cockroachdb/cockroach/protobuf/src/google/protobuf/stubs/status.cc:95
    #3 0x7f36a496106b in __cxx_global_var_init.1 /go/src/github.com/cockroachdb/cockroach/protobuf/src/google/protobuf/stubs/status.cc:86:34
    #4 0x7f36a496106b in _GLOBAL__sub_I_status.tmp.4a08e781b33b.11548.ii /root/.ccache/tmp/status.tmp.4a08e781b33b.11548.ii
    #5 0x7f36a53669f9  (/lib64/ld-linux-x86-64.so.2+0xe9f9)
    #6 0x7f36a5366ae2  (/lib64/ld-linux-x86-64.so.2+0xeae2)
    #7 0x7f36a53591c9  (/lib64/ld-linux-x86-64.so.2+0x11c9)

  Uninitialized value was created by an allocation of 'ref.tmp' in the stack frame of function '_ZN6google8protobuf4util6StatusC2ENS1_5error4CodeENS0_11StringPieceE'
    #0 0x7f36a4973400 in google::protobuf::util::Status::Status(google::protobuf::util::error::Code, google::protobuf::StringPiece) /go/src/github.com/cockroachdb/cockroach/protobuf/src/google/protobuf/stubs/status.cc:93

SUMMARY: MemorySanitizer: use-of-uninitialized-value /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/basic_string.h:240:8 in std::string::_Rep::_M_dispose(std::allocator<char> const&)

[tamird]

Further investigation suggests that I'm not properly persuading my clang to use the instrumented libstdc++ with which it ships, and it's using the system's copy, instead.

@dvyukov: @andreimatei suggested I ask you - what is the magic invocation of clang that will have it use a specific libstdc++.so?
Recapping from earlier in this issue, I'm using Chromium's prebuilt clang:

$ clang --version
clang version 3.8.0 (trunk 255169)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /third_party/llvm-build/Release+Asserts/bin

which shipped with /third_party/llvm-build/Release+Asserts/lib/libstdc++.so.6

Any help here would be more than appreciated!

[bdarnell]

For the record: to run any c++ program with msan, you need a custom build of libc++, as documented in this page.

With this done, programs using protobufs are msan-clean, so no action is necessary on the protobuf side and this issue can be closed.