Unable to build with jemalloc on MacOS X

[adamretter]

After installing jemalloc on MacOS with Homebrew:

$ brew install jemalloc

I am no longer able to build a release with make rocksdbjavastatic. I now get this error at the linker stage:

	  libz.a libbz2.a libsnappy.a liblz4.a libzstd.a 
Undefined symbols for architecture x86_64:
  "_malloc_stats_print", referenced from:
      rocksdb::DumpMallocStats(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) in malloc_stats.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [rocksdbjavastatic] Error 1

[adamretter]

@sagar0 Any progress on investigating this please?

[andrewarrow]

i got there too from:

~/os/rocksdb/examples $ cd ..
~/os/rocksdb $ make static_lib
  GEN      util/build_version.cc
  AR       librocksdb.a
ar: creating archive librocksdb.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: librocksdb.a(rocks_lua_compaction_filter.o) has no symbols
~/os/rocksdb $ cd examples/
~/os/rocksdb/examples $ make all
cd .. && /Applications/Xcode.app/Contents/Developer/usr/bin/make static_lib
  GEN      util/build_version.cc
make[1]: Nothing to be done for `static_lib'.
g++ -fno-rtti simple_example.cc -osimple_example ../librocksdb.a -I../include -O2 -std=c++11 -lz -lbz2 -std=c++11  -DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX  -DOS_MACOSX -DZLIB -DBZIP2 -DROCKSDB_BACKTRACE -Wshorten-64-to-32 -march=native  -DHAVE_SSE42 -DROCKSDB_SUPPORT_THREAD_LOCAL -DROCKSDB_JEMALLOC -DJEMALLOC_NO_DEMANGLE    -lpthread
Undefined symbols for architecture x86_64:
  "_malloc_stats_print", referenced from:
      rocksdb::DumpMallocStats(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) in librocksdb.a(malloc_stats.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [simple_example] Error 1

[iKenApple]

@adamretter I got this error too.
brew install rocksdb success, but when i use librocksdb.a，it reported:

Undefined symbols for architecture x86_64:
  "_malloc_stats_print", referenced from:
      rocksdb::DumpMallocStats(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) in librocksdb.a(malloc_stats.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [redis-server] Error 1

[iKenApple]

@adamretter I add include and lib of jemalloc in my Makefile, then error solved.

[sagar0]

I started looking into this issue today. Here are my findings so far:

• make rocksdbjavastatic works fine when jemalloc is not installed on OS X .
• make rocksdbjavastatic barfs with the error mentioned above by Adam when jemalloc is installed via brew install jemalloc.
• This is not something new that got introduced with Use jemalloc in rocksdbjni library built via vagrant #2916. In Use jemalloc in rocksdbjni library built via vagrant #2916 I made the changes only to the linux side of things, and Linux seems to find jemalloc lib and include paths automatically. Automatic-finding of jemalloc install paths seems to break down on OS X when installed with brew; may be they need to be configured explicitly.
• I was able to repro the issue on 65a9cd6, the parent commit of 96a13b4 as well.

[adamretter]

@sagar0 I am experiencing this problem again, I think I have a solution, will send a PR shortly.

[adamretter]

Okay I managed to enable the build (in PR #4767) to complete successfully with jealloc on macOS, but actually using RocksJava with jemalloc eventually causes the tests to crash, i.e. running make -j4 jtest will lead to this:

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000125ff9fbd, pid=39108, tid=0x0000000000002703
#
# JRE version: OpenJDK Runtime Environment (8.0_192-b01) (build 1.8.0_192-b01)
# Java VM: OpenJDK 64-Bit Server VM (25.192-b01 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C  [libjemalloc.2.dylib+0x5fbd]  free+0x94

...

Stack: [0x000070000599a000,0x0000700005a9a000],  sp=0x0000700005a99c10,  free space=1023k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libjemalloc.2.dylib+0x5fbd]  free+0x94
C  [libjemalloc.2.dylib+0x39cd0]  zone_free+0xaf
C  [CoreFoundation+0x13f2cb]  _CFRelease+0x2c4
C  [SystemConfiguration+0x6c84]  __SCThreadSpecificDataFinalize+0x2a
C  [libsystem_pthread.dylib+0x36ab]  _pthread_tsd_cleanup+0x1db
C  [libsystem_pthread.dylib+0x66b1]  _pthread_exit+0x46
C  [libsystem_pthread.dylib+0x3344]  _pthread_body+0x89
C  [libsystem_pthread.dylib+0x62a7]  _pthread_start+0x46
C  [libsystem_pthread.dylib+0x2445]  thread_start+0xd

See attached:

hs_err_pid39108.log

[edtbl76]

Did anyone find a resolution w/ running the tests? When running tests or any code against an OSX built JNI, I get the same SIGSEGVs.

The PR change to build-tools doesn't resolve the issue.