-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrong dynamic library linking order when compiling with MKL FFT libraries #958
Comments
Did you source The _vars script sets Which compiler are you using? |
Sorry for the late reply. I am compiling inside an automated build environment (Gentoo ebuild). MKL is a system library and is correctly detected by cmake (not using any helper scripts such as mkl_vars.sh). I am using gcc: With best regards, |
Where is it located? |
/opt/intel/oneapi/mkl/2022.2.1 The following environmental variables are set system-wide (/etc/env.d/70intel-mkl): |
I don't think these variables fully mimic what |
There is /opt/intel/oneapi/mkl/2022.2.1/env/vars.sh (attached) The simple patch I provided with my bug report solved all issues for me and I don't have to rely on any additional scripts. I thought I would share it with the community. |
There is existing FFTW_LIBRARIES link like below. relion/src/apps/CMakeLists.txt Lines 342 to 346 in ec417f5
Could you (@markus-meier74 ) provide the below information to reproduce your problem?
|
@do-jason: That's correct, but the statement that the relion libraries ("relion_lib") also depend on ${FFTW_LIBRARIES} is missed, because this statement is inside a if(NOT MKLFFT) ... endif(NOT MKLFFT) block: relion/src/apps/CMakeLists.txt Lines 222 to 230 in ec417f5
Just move "target_link_libraries(relion_lib ${FFTW_LIBRARIES})" outside this block and everything works as it should. Will provide the requested info as soon as it can be contrieved. |
When we link to MKLFFT, our assumption is that MKL include/linking paths set by the vars script are enough and that |
@biochem-fan: Not correct, as your colleague do-jason just pointed out, your CMakeLists.txt does use FFTW_LIBRARIES for linking with MKL, on line 345: relion/src/apps/CMakeLists.txt Lines 342 to 346 in ec417f5
So it is a Relion bug in my opinion. The vars script is unlikely to set the correct order of the libraries as it works outside cmake, and this linking error may randomly occur. |
I did some more investigating. I get the same linking error even if I source /opt/intel/oneapi/mkl/2022.2.1/env/vars.sh before building. The bug is triggered by the "--as-needed" linker flag which is appended by default in the Gentoo ebuild environment. Here are the steps to reproduce the error on your system (bash script) and all the information you were asking for: cmake configuration log: Ninja build log: CMakeCache: Build environment: System informatioin: Libraries with version info: gcc -v |
@biochem-fan @markus-meier74 I have tested it and found that the problem occurs when shared library build is on (-DBUILD_SHARED_LIBS=ON) and MKL is enabled for non-Intel compiler. FFTW_LIBRARIES is set by cmake when own FFTW is needed or MKL is enabled without Intel compiler. I have tested it with Intel compiler with MKL, and the given patch also worked fine. target_link_libraries(relion_lib ${FFTW_LIBRARIES})
if(NOT MKLFFT)
if(BUILD_OWN_FFTW)
add_dependencies(relion_lib own_fftw_lib)
endif()
if(BUILD_OWN_FFTWF)
add_dependencies(relion_lib own_fftwf_lib)
endif()
endif(NOT MKLFFT) |
linker flag (e.g. in Gentoo), as reported in issue #958. Thanks to @markus-meier74 and @do-jason for the report and investigation.
Thank you very much for investigation. I understood. |
Hi, I get the following linking error when compiling Relion with -DMKLFFT=ON:
FAILED: bin/relion_display
: && /usr/bin/mpicxx -march=native -mfpmath=sse -O3 -pipe -DNDEBUG -fopenmp -std=c++11 -Wl,-O1 -Wl,--as-needed -rdynamic src/apps/CMakeFiles/display.dir/display.cpp.o -o bin/relion_display -L/opt/intel/oneapi/mkl/2022.2.1/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core /opt/cuda/lib64/libcufft.so /opt/cuda/lib64/libcufft.so -lmpicxx -lmpi -ldl lib64/librelion_gui_lib.so /usr/lib64/fltk/libfltk_images.so /usr/lib64/fltk/libfltk_forms.so /usr/lib64/fltk/libfltk.so -lSM -lICE -lX11 -lXext -lm -ltiff lib64/librelion_lib.so lib64/librelion_gpu_util.so /opt/cuda/lib64/libcurand.so /opt/cuda/lib64/libcufft.so -ltiff lib64/librelion_jaz_gpu_util.so /opt/cuda/lib64/libcudart_static.a -ldl -Wl,-Bstatic -lrt -Wl,-Bdynamic -lpng -ljpeg && :
/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_init_threads' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_plan_dft_r2c'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_plan_dft_c2r' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_gui_lib.so: undefined reference to
fftw_malloc'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_execute_dft_r2c' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_plan_dft_r2c'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_gui_lib.so: undefined reference to
fftw_free' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_execute_dft_c2r'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_execute_dft_c2r' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_plan_with_nthreads'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_cleanup' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_execute_dft_r2c'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_plan_dft' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_cleanup_threads'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftwf_plan_dft_c2r' /usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to
fftw_destroy_plan'/usr/lib/gcc/x86_64-pc-linux-gnu/11/../../../../x86_64-pc-linux-gnu/bin/ld: lib64/librelion_lib.so: undefined reference to `fftwf_destroy_plan'
collect2: error: ld returned 1 exit status
The reason is cyclic dependency. librelion_lib.so also depends on libmkl_intel_lp64.so, but -lmkl_intel_lp64 is given before lib64/librelion_lib.so on the command line.
Here is a patch to src/apps/CMakeLists.txt that puts the libraries into the right order and fixed the problem for me:
relion_MKLFFT.patch.txt
With best regards,
Markus
The text was updated successfully, but these errors were encountered: