Linking fails with various undefined reference to *AARCH64* (x86_64 build on x86_64)

[frantisekz]

I am trying to build opencl-clang as standalone library with:

/usr/bin/cmake -S . -B redhat-linux-build -DCMAKE_C_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_Fortran_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_INSTALL_DO_STRIP:BOOL=OFF -DCMAKE_INSTALL_PREFIX:PATH=/usr -DINCLUDE_INSTALL_DIR:PATH=/usr/include -DLIB_INSTALL_DIR:PATH=/usr/lib64 -DSYSCONF_INSTALL_DIR:PATH=/etc -DSHARE_INSTALL_PREFIX:PATH=/usr/share -DLIB_SUFFIX=64 -DBUILD_SHARED_LIBS:BOOL=ON -DPREFERRED_LLVM_VERSION=16.0.0

I have #415 and #416 applied on top of the 16.0.0 tag here to be able to even start the build process. spirv-llvm-translator is taken from system (separate .so, not part of llvm). The linking fails with:

[ 98%] Linking CXX shared library libopencl-clang.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/opencl-clang.dir/link.txt --verbose=1
/usr/bin/clang++ -fPIC -O2 -flto=thin -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS --config /usr/lib/rpm/redhat/redhat-hardened-clang.cfg -fstack-protector-strong   -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -fno-rtti  -Wl,--no-undefined -Wl,-Bsymbolic -Wl,--version-script=/builddir/build/BUILD/opencl-clang-1adc46ef38aeba1585b3587b95b6af692a12e4f4/opencl_clang.map -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now   -flto=thin -fno-openmp-implicit-rpath -Wl,--build-id=sha1 -shared -Wl,-soname,libopencl-clang.so -o libopencl-clang.so "CMakeFiles/opencl-clang.dir/opencl_clang.cpp.o" "CMakeFiles/opencl-clang.dir/options.cpp.o" "CMakeFiles/opencl-clang.dir/pch_mgr.cpp.o" "CMakeFiles/opencl-clang.dir/options_compile.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c.h.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-base.h.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-12-spir.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-20-spir.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-30-spir.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-12-spir64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-20-spir64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-30-spir64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-12-spir-fp64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-20-spir-fp64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-30-spir-fp64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-12-spir64-fp64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-20-spir64-fp64.mod.cpp.o" "cl_headers/CMakeFiles/cl_headers.dir/opencl-c-30-spir64-fp64.mod.cpp.o" cl_headers/CMakeFiles/cl_headers.dir/module.modulemap.cpp.o  -ldl -lLLVMSPIRVLib -lclang-cpp /usr/lib64/libLLVMX86CodeGen.a /usr/lib64/libLLVMX86AsmParser.a /usr/lib64/libLLVMX86Desc.a /usr/lib64/libLLVMX86Info.a /usr/lib64/libLLVMX86Disassembler.a /usr/lib64/libLLVMAnalysis.a /usr/lib64/libLLVMCodeGen.a /usr/lib64/libLLVMCore.a /usr/lib64/libLLVMipo.a /usr/lib64/libLLVMInstCombine.a /usr/lib64/libLLVMInstrumentation.a /usr/lib64/libLLVMMC.a /usr/lib64/libLLVMMCParser.a /usr/lib64/libLLVMObjCARCOpts.a /usr/lib64/libLLVMOption.a /usr/lib64/libLLVMScalarOpts.a /usr/lib64/libLLVMSupport.a /usr/lib64/libLLVMTransformUtils.a /usr/lib64/libLLVMVectorize.a /usr/lib64/libLLVMAsmPrinter.a /usr/lib64/libLLVMSelectionDAG.a /usr/lib64/libLLVMMCDisassembler.a /usr/lib64/libLLVMProfileData.a /usr/lib64/libLLVMObject.a /usr/lib64/libLLVMBitWriter.a /usr/lib64/libLLVMIRReader.a /usr/lib64/libLLVMAsmParser.a /usr/lib64/libLLVMTarget.a /usr/lib64/libLLVMBitReader.a /usr/lib64/libLLVMNVPTXCodeGen.a /usr/lib64/libLLVMNVPTXDesc.a /usr/lib64/libLLVMNVPTXInfo.a /usr/lib64/libLLVMAMDGPUCodeGen.a /usr/lib64/libLLVMAMDGPUAsmParser.a /usr/lib64/libLLVMAMDGPUDesc.a /usr/lib64/libLLVMAMDGPUInfo.a /usr/lib64/libLLVMCFGuard.a /usr/lib64/libLLVMX86Info.a /usr/lib64/libLLVMAsmPrinter.a /usr/lib64/libLLVMGlobalISel.a /usr/lib64/libLLVMSelectionDAG.a /usr/lib64/libLLVMPasses.a /usr/lib64/libLLVMCoroutines.a /usr/lib64/libLLVMipo.a /usr/lib64/libLLVMInstrumentation.a /usr/lib64/libLLVMVectorize.a /usr/lib64/libLLVMFrontendOpenMP.a /usr/lib64/libLLVMLinker.a /usr/lib64/libLLVMIRPrinter.a /usr/lib64/libLLVMMIRParser.a /usr/lib64/libLLVMCodeGen.a /usr/lib64/libLLVMObjCARCOpts.a /usr/lib64/libLLVMScalarOpts.a /usr/lib64/libLLVMInstCombine.a /usr/lib64/libLLVMAggressiveInstCombine.a /usr/lib64/libLLVMTransformUtils.a /usr/lib64/libLLVMBitWriter.a /usr/lib64/libLLVMTarget.a /usr/lib64/libLLVMAMDGPUUtils.a /usr/lib64/libLLVMAnalysis.a /usr/lib64/libLLVMProfileData.a /usr/lib64/libLLVMSymbolize.a /usr/lib64/libLLVMDebugInfoDWARF.a /usr/lib64/libLLVMDebugInfoPDB.a /usr/lib64/libLLVMDebugInfoMSF.a /usr/lib64/libLLVMObject.a /usr/lib64/libLLVMMCParser.a /usr/lib64/libLLVMIRReader.a /usr/lib64/libLLVMAsmParser.a /usr/lib64/libLLVMBitReader.a /usr/lib64/libLLVMTextAPI.a /usr/lib64/libLLVMCore.a /usr/lib64/libLLVMRemarks.a /usr/lib64/libLLVMBitstreamReader.a /usr/lib64/libLLVMMC.a /usr/lib64/libLLVMDebugInfoCodeView.a /usr/lib64/libLLVMBinaryFormat.a /usr/lib64/libLLVMTargetParser.a /usr/lib64/libLLVMSupport.a /usr/lib64/libLLVMDemangle.a -lrt -ldl -lm /usr/lib64/libz.so /usr/lib64/libtinfo.so 
/usr/bin/ld: /tmp/lto-llvm-ead063.o: in function `llvm::InitializeAllTargetInfos()':
/usr/include/llvm/Config/Targets.def:26: undefined reference to `LLVMInitializeAArch64TargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:28: undefined reference to `LLVMInitializeARMTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:29: undefined reference to `LLVMInitializeAVRTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:30: undefined reference to `LLVMInitializeBPFTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:31: undefined reference to `LLVMInitializeHexagonTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:32: undefined reference to `LLVMInitializeLanaiTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:33: undefined reference to `LLVMInitializeLoongArchTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:34: undefined reference to `LLVMInitializeMipsTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:35: undefined reference to `LLVMInitializeMSP430TargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:37: undefined reference to `LLVMInitializePowerPCTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:38: undefined reference to `LLVMInitializeRISCVTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:39: undefined reference to `LLVMInitializeSparcTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:40: undefined reference to `LLVMInitializeSystemZTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:41: undefined reference to `LLVMInitializeVETargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:42: undefined reference to `LLVMInitializeWebAssemblyTargetInfo'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:44: undefined reference to `LLVMInitializeXCoreTargetInfo'
/usr/bin/ld: /tmp/lto-llvm-ead063.o: in function `llvm::InitializeAllTargets()':
/usr/include/llvm/Config/Targets.def:26: undefined reference to `LLVMInitializeAArch64Target'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:28: undefined reference to `LLVMInitializeARMTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:29: undefined reference to `LLVMInitializeAVRTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:30: undefined reference to `LLVMInitializeBPFTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:31: undefined reference to `LLVMInitializeHexagonTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:32: undefined reference to `LLVMInitializeLanaiTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:33: undefined reference to `LLVMInitializeLoongArchTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:34: undefined reference to `LLVMInitializeMipsTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:35: undefined reference to `LLVMInitializeMSP430Target'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:37: undefined reference to `LLVMInitializePowerPCTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:38: undefined reference to `LLVMInitializeRISCVTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:39: undefined reference to `LLVMInitializeSparcTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:40: undefined reference to `LLVMInitializeSystemZTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:41: undefined reference to `LLVMInitializeVETarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:42: undefined reference to `LLVMInitializeWebAssemblyTarget'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:44: undefined reference to `LLVMInitializeXCoreTarget'
/usr/bin/ld: /tmp/lto-llvm-ead063.o: in function `llvm::InitializeAllAsmPrinters()':
/usr/include/llvm/Config/AsmPrinters.def:27: undefined reference to `LLVMInitializeAArch64AsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:29: undefined reference to `LLVMInitializeARMAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:30: undefined reference to `LLVMInitializeAVRAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:31: undefined reference to `LLVMInitializeBPFAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:32: undefined reference to `LLVMInitializeHexagonAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:33: undefined reference to `LLVMInitializeLanaiAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:34: undefined reference to `LLVMInitializeLoongArchAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:35: undefined reference to `LLVMInitializeMipsAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:36: undefined reference to `LLVMInitializeMSP430AsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:38: undefined reference to `LLVMInitializePowerPCAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:39: undefined reference to `LLVMInitializeRISCVAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:40: undefined reference to `LLVMInitializeSparcAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:41: undefined reference to `LLVMInitializeSystemZAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:42: undefined reference to `LLVMInitializeVEAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:43: undefined reference to `LLVMInitializeWebAssemblyAsmPrinter'
/usr/bin/ld: /usr/include/llvm/Config/AsmPrinters.def:45: undefined reference to `LLVMInitializeXCoreAsmPrinter'
/usr/bin/ld: /tmp/lto-llvm-ead063.o: in function `llvm::InitializeAllAsmParsers()':
/usr/include/llvm/Config/AsmParsers.def:27: undefined reference to `LLVMInitializeAArch64AsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:29: undefined reference to `LLVMInitializeARMAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:30: undefined reference to `LLVMInitializeAVRAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:31: undefined reference to `LLVMInitializeBPFAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:32: undefined reference to `LLVMInitializeHexagonAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:33: undefined reference to `LLVMInitializeLanaiAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:34: undefined reference to `LLVMInitializeLoongArchAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:35: undefined reference to `LLVMInitializeMipsAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:36: undefined reference to `LLVMInitializeMSP430AsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:37: undefined reference to `LLVMInitializePowerPCAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:38: undefined reference to `LLVMInitializeRISCVAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:39: undefined reference to `LLVMInitializeSparcAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:40: undefined reference to `LLVMInitializeSystemZAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:41: undefined reference to `LLVMInitializeVEAsmParser'
/usr/bin/ld: /usr/include/llvm/Config/AsmParsers.def:42: undefined reference to `LLVMInitializeWebAssemblyAsmParser'
/usr/bin/ld: /tmp/lto-llvm-ead063.o: in function `llvm::InitializeAllTargetMCs()':
/usr/include/llvm/Config/Targets.def:26: undefined reference to `LLVMInitializeAArch64TargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:28: undefined reference to `LLVMInitializeARMTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:29: undefined reference to `LLVMInitializeAVRTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:30: undefined reference to `LLVMInitializeBPFTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:31: undefined reference to `LLVMInitializeHexagonTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:32: undefined reference to `LLVMInitializeLanaiTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:33: undefined reference to `LLVMInitializeLoongArchTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:34: undefined reference to `LLVMInitializeMipsTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:35: undefined reference to `LLVMInitializeMSP430TargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:37: undefined reference to `LLVMInitializePowerPCTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:38: undefined reference to `LLVMInitializeRISCVTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:39: undefined reference to `LLVMInitializeSparcTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:40: undefined reference to `LLVMInitializeSystemZTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:41: undefined reference to `LLVMInitializeVETargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:42: undefined reference to `LLVMInitializeWebAssemblyTargetMC'
/usr/bin/ld: /usr/include/llvm/Config/Targets.def:44: undefined reference to `LLVMInitializeXCoreTargetMC'
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

[frantisekz]

I was able to work around this by:

---
 CMakeLists.txt   | 2 +-
 opencl_clang.cpp | 8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3d7eff1..d4713a6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -341,7 +341,7 @@ target_link_libraries( ${TARGET_NAME}
                        LLVMX86Info
                        LLVMX86Disassembler
                        LLVMAnalysis
-                       LLVMCodeGen
+                       LLVMX86CodeGen
                        LLVMCore
                        LLVMipo
                        LLVMInstCombine
diff --git a/opencl_clang.cpp b/opencl_clang.cpp
index eed8f02..930cc11 100644
--- a/opencl_clang.cpp
+++ b/opencl_clang.cpp
@@ -99,10 +99,10 @@ void CommonClangInitialize() {
       // llvm_shutdown before static object are destroyed, so we use atexit to
       // satisfy this requirement.
       atexit(CommonClangTerminate);
-      llvm::InitializeAllTargets();
-      llvm::InitializeAllAsmPrinters();
-      llvm::InitializeAllAsmParsers();
-      llvm::InitializeAllTargetMCs();
+      llvm::InitializeNativeTarget();
+      llvm::InitializeNativeTargetAsmPrinter();
+      llvm::InitializeNativeTargetAsmParser();
+      //llvm::InitializeAllTargetMCs();
       lazyCCInit = false;
     }
   }
-- 
2.40.0

[anbe42]

This is fallout caused by switching from add_llvm_library() to add_library() (commit 743bd15).

Previously that was add_llvm_library(... LINK_COMPONENTS all ...) which used the shared library /usr/lib/llvm-15/lib/libLLVM-15.so.1, but now it links with the individual static libraries libLLVM*.a even in case of USE_PREBUILT_LLVM and LLVM_LINK_LLVM_DYLIB. I couldn't convince cmake to use the shared library nevertheless with this new setup.

My guess is that the component list given to target_link_libraries() is either incomplete or incorrectly ordered resulting in these missing symbols. (It may work on overzealous linkers that don't strictly adhere to order of the libraries given.)

[wenju-he]

Address this issue in #439

[wenju-he]

#439 is merged into ocl-open-160 branch, @frantisekz @anbe42 could you please help to check if this issue is resolved?

[frantisekz]

Yes, the issue is resolved, thanks!