PyTorch master failed to build with glog 0.5.0.

[xkszltl]

Bug

Log:

root@164a9000a4ab:/tmp/scratch/pytorch/build# cmake --build . -- -j1
[1/153] : && /usr/bin/g++-8 -fdebug-prefix-map='/tmp/scratch'='/usr/local/src' -g -march=haswell -mtune=generic -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic c10/cuda/test/CMakeFiles/c10_cuda_CUDATest.dir/impl/CUDATest.cpp.o -o bin/c10_cuda_CUDATest  -Wl,-rpath,/tmp/scratch/pytorch/build/lib:/usr/local/lib:/usr/local/cuda/lib64:  lib/libc10_cuda.so  lib/libgtest_main.a  lib/libc10.so  /usr/local/lib/libglog.so.0.5.0  /usr/local/lib/libgflags.so.2.2.2  -lpthread  /usr/local/cuda/lib64/libcudart.so  lib/libgtest.a  -pthread && :
FAILED: bin/c10_cuda_CUDATest 
: && /usr/bin/g++-8 -fdebug-prefix-map='/tmp/scratch'='/usr/local/src' -g -march=haswell -mtune=generic -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic c10/cuda/test/CMakeFiles/c10_cuda_CUDATest.dir/impl/CUDATest.cpp.o -o bin/c10_cuda_CUDATest  -Wl,-rpath,/tmp/scratch/pytorch/build/lib:/usr/local/lib:/usr/local/cuda/lib64:  lib/libc10_cuda.so  lib/libgtest_main.a  lib/libc10.so  /usr/local/lib/libglog.so.0.5.0  /usr/local/lib/libgflags.so.2.2.2  -lpthread  /usr/local/cuda/lib64/libcudart.so  lib/libgtest.a  -pthread && :
lib/libc10.so: undefined reference to `google::glog_internal_namespace_::IsGoogleLoggingInitialized()'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

glog 0.5 is released a few days ago.
IIRC pytorch redefined that func to expose it.
Wondering that's related.

Environment

• PyTorch Version (e.g., 1.0): master
• OS (e.g., Linux): Ubuntu 18.04 / CentOS 7
• How you installed PyTorch (conda, pip, source): source
• Build command you used (if compiling from source): cmake+ninja+gcc
• Python version: 3.6
• CUDA/cuDNN version: 11.3

cc @malfet @seemethere @walterddr

[xkszltl]

Symbol binding has changed from GLOBAL to LOCAL.

glog 0.4.0:

# readelf -a /usr/local/lib/libglog.so.0.4.0 | grep IsGoogleLoggingInitialized
0000000000222538  000000fa00000007 R_X86_64_JUMP_SLOT     0000000000019800 _ZN6google24glog_internal_namespace_26IsGoogleLoggingInitializedEv + 0
   250: 0000000000019800    12 FUNC    GLOBAL DEFAULT    12 _ZN6google24glog_internal_namespace_26IsGoogleLoggingInitializedEv
   597: 0000000000019800    12 FUNC    GLOBAL DEFAULT    12 _ZN6google24glog_internal_namespace_26IsGoogleLoggingInitializedEv

glog 0.5.0:

# readelf -a /usr/local/lib/libglog.so.0.5.0 | grep IsGoogleLoggingInitialized
   331: 000000000001b4d0    12 FUNC    LOCAL  DEFAULT    12 _ZN6google24glog_internal_namespace_26IsGoogleLoggingInitializedEv

[xkszltl]

This commit sets default visibility to hidden and could be related:
google/glog@c30896d#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a

[mruberry]

What about PyTorch 1.8, @xkszltl?

[xkszltl]

Haven't tried but I don't think it is related to change in pytorch.
If it's hidden now, you won't be able to take it out like this:

pytorch/c10/util/Logging.cpp

Lines 194 to 202 in 1e77ba3

	// Google glog's api does not have an external function that allows one to check
	// if glog is initialized or not. It does have an internal function - so we are
	// declaring it here. This is a hack but has been used by a bunch of others too
	// (e.g. Torch).
	namespace google {
	namespace glog_internal_namespace_ {
	bool IsGoogleLoggingInitialized();
	} // namespace glog_internal_namespace_
	} // namespace google

[xkszltl]

I've added an API to glog master so that you don't have to use the previous hack:
google/glog#651
But you'll need some mechanism to detect its existence.

[malfet]

Thank you for reporting. We would accept the patch that fixes it

[xkszltl]

Given they're different functions (named the same but old one is in google::glog_internal_namespace_ and the new one is in google directly), I don't know how to automatically route to the correct one.
Any ideas?

[essandess]

This appears to be the fix: google/glog#125 (comment)

Related:

• Undefined symbols to build or run Pytorch on macOS google/glog#660
• https://trac.macports.org/ticket/62921

[xkszltl]

@essandess
No, that's exactly what doesn't work in glog 0.5.0
See the linked PR: google/glog#651 in your issue.

• I added a public API in master, not 0.5.
• They don't want to keep internal API for compatibility.
• Need a fancy way to distinguish between 0.4/0.5/master and generate different code.

[essandess]

This remains an issue in torch version 1.9.1:

:info:build : && /opt/local/var/macports/build/_opt_local_ports_python_py-pytorch/py39-pytorch/work/compwrap/cxx/usr/bin/clang++ -arch x86_64 -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -fvisibility-inlines-hidden -Wno-deprecated-declarations -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-typedef-redefinition -Wno-unknown-warning-option -Wno-unused-private-field -Wno-inconsistent-missing-override -Wno-aligned-allocation-unavailable -Wno-c++14-extensions -Wno-constexpr-not-const -Wno-missing-braces -Qunused-arguments -fcolor-diagnostics -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-unused-private-field -Wno-missing-braces -Wno-c++14-extensions -Wno-constexpr-not-const -O3 -DNDEBUG -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -mmacosx-version-min=11.0 -dynamiclib -Wl,-headerpad_max_install_names -arch x86_64 -rdynamic -o lib/libc10.dylib -install_name @rpath/libc10.dylib c10/CMakeFiles/c10.dir/core/Allocator.cpp.o c10/CMakeFiles/c10.dir/core/CPUAllocator.cpp.o c10/CMakeFiles/c10.dir/core/CopyBytes.cpp.o c10/CMakeFiles/c10.dir/core/DefaultDtype.cpp.o c10/CMakeFiles/c10.dir/core/Device.cpp.o c10/CMakeFiles/c10.dir/core/DeviceType.cpp.o c10/CMakeFiles/c10.dir/core/DispatchKey.cpp.o c10/CMakeFiles/c10.dir/core/DispatchKeySet.cpp.o c10/CMakeFiles/c10.dir/core/GeneratorImpl.cpp.o c10/CMakeFiles/c10.dir/core/GradMode.cpp.o c10/CMakeFiles/c10.dir/core/InferenceMode.cpp.o c10/CMakeFiles/c10.dir/core/Scalar.cpp.o c10/CMakeFiles/c10.dir/core/Storage.cpp.o c10/CMakeFiles/c10.dir/core/StorageImpl.cpp.o c10/CMakeFiles/c10.dir/core/Stream.cpp.o c10/CMakeFiles/c10.dir/core/TensorImpl.cpp.o c10/CMakeFiles/c10.dir/core/TensorOptions.cpp.o c10/CMakeFiles/c10.dir/core/UndefinedTensorImpl.cpp.o c10/CMakeFiles/c10.dir/core/impl/DeviceGuardImplInterface.cpp.o c10/CMakeFiles/c10.dir/core/impl/LocalDispatchKeySet.cpp.o c10/CMakeFiles/c10.dir/core/impl/SizesAndStrides.cpp.o c10/CMakeFiles/c10.dir/core/thread_pool.cpp.o c10/CMakeFiles/c10.dir/mobile/CPUCachingAllocator.cpp.o c10/CMakeFiles/c10.dir/mobile/CPUProfilingAllocator.cpp.o c10/CMakeFiles/c10.dir/util/Array.cpp.o c10/CMakeFiles/c10.dir/util/Backtrace.cpp.o c10/CMakeFiles/c10.dir/util/C++17.cpp.o c10/CMakeFiles/c10.dir/util/DeadlockDetection.cpp.o c10/CMakeFiles/c10.dir/util/Exception.cpp.o c10/CMakeFiles/c10.dir/util/Half.cpp.o c10/CMakeFiles/c10.dir/util/LeftRight.cpp.o c10/CMakeFiles/c10.dir/util/Logging.cpp.o c10/CMakeFiles/c10.dir/util/MathConstants.cpp.o c10/CMakeFiles/c10.dir/util/Metaprogramming.cpp.o c10/CMakeFiles/c10.dir/util/Optional.cpp.o c10/CMakeFiles/c10.dir/util/SmallVector.cpp.o c10/CMakeFiles/c10.dir/util/StringUtil.cpp.o c10/CMakeFiles/c10.dir/util/ThreadLocalDebugInfo.cpp.o c10/CMakeFiles/c10.dir/util/Type.cpp.o c10/CMakeFiles/c10.dir/util/TypeList.cpp.o c10/CMakeFiles/c10.dir/util/TypeTraits.cpp.o c10/CMakeFiles/c10.dir/util/Unicode.cpp.o c10/CMakeFiles/c10.dir/util/UniqueVoidPtr.cpp.o c10/CMakeFiles/c10.dir/util/complex_math.cpp.o c10/CMakeFiles/c10.dir/util/flags_use_gflags.cpp.o c10/CMakeFiles/c10.dir/util/flags_use_no_gflags.cpp.o c10/CMakeFiles/c10.dir/util/intrusive_ptr.cpp.o c10/CMakeFiles/c10.dir/util/numa.cpp.o c10/CMakeFiles/c10.dir/util/signal_handler.cpp.o c10/CMakeFiles/c10.dir/util/thread_name.cpp.o c10/CMakeFiles/c10.dir/util/typeid.cpp.o  /opt/local/lib/libglog.0.5.0.dylib  /opt/local/lib/libgflags.2.2.2.dylib && :
:info:build ld: warning: dylib (/opt/local/lib/libglog.0.5.0.dylib) was built for newer macOS version (11.3) than being linked (11.0)
:info:build Undefined symbols for architecture x86_64:
:info:build   "google::glog_internal_namespace_::IsGoogleLoggingInitialized()", referenced from:
:info:build       c10::InitCaffeLogging(int*, char**) in Logging.cpp.o

[essandess]

This remains an issue in torch version 1.10.1:

:info:build Undefined symbols for architecture x86_64:
:info:build   "google::glog_internal_namespace_::IsGoogleLoggingInitialized()", referenced from:
:info:build       c10::InitCaffeLogging(int*, char**) in Logging.cpp.o
:info:build ld: symbol(s) not found for architecture x86_64