Intel MKL FATAL ERROR: Cannot load libmkl_avx512.so or libmkl_def.so

[RenieWell]

I follow your advice and create virtual envirment by conda create -n trefide python=3.6
and I have installed with pip successfully.
However when I ran the code in demo file Demo PMD Compression & Denoising.ipynb
in the line

spatial_thresh, temporal_thresh = determine_thresholds((fov_height, fov_width, num_frames),
(block_height, block_width),
consec_failures, max_iters_main,
max_iters_init, tol,
d_sub, t_sub, 5, True)

the iPython crashed down and restarted. The error in console is

Intel MKL FATAL ERROR: Cannot load libmkl_avx512.so or libmkl_def.so

I follow the advice in https://stackoverflow.com/questions/36659453/intel-mkl-fatal-error-cannot-load-libmkl-avx2-so-or-libmkl-def-so by entering the commond conda install nomkl numpy scipy scikit-learn numexpr conda remove mkl mkl-service, but it didn't work.
As I have install the MTK, I tried to locate the libmkl_avx512.so, which is under the path /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64_lin
So, I modified the makefile as

CXX = icpc
CXXFLAGS = -mkl=sequential -qopenmp -O3 -fPIC -I$(shell pwd)/proxtv -I$(shell pwd)/glmgen/include/ -D NOMATLAB=1 #-mkl=parallel
LDFLAGS = -shared
PROXTV = proxtv
LIBPROXTV = $(PROXTV)/libproxtv.so
GLMGEN = glmgen
MKLLIB = /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64
LIBGLMGEN = $(GLMGEN)/lib/libglmgen.so
LIBTREFIDE = libtrefide.so
LIBS = -lmkl_intel_lp64 -lmkl_core -lm -lmkl_intel_thread -liomp5
#LIBS = /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64
SRCS = utils/welch.cpp proxtf/wpdas.cpp proxtf/line_search.cpp proxtf/utils.cpp proxtf/l1tf/ipm.cpp proxtf/admm.cpp pmd/pmd.cpp pmd/decimation.cpp
OBJS = $(patsubst %.cpp,%.o,$(SRCS))
.PHONY: all
all: $(LIBPROXTV) $(LIBGLMGEN) $(LIBTREFIDE) $(MKLLIB)
$(LIBPROXTV):
cd $(PROXTV); make;
$(LIBGLMGEN):
cd $(GLMGEN); make;
$(LIBTREFIDE): $(OBJS)
$(CXX) $(CXXFLAGS) $(LDFLAGS) -o $@ $^ $(LIBS)
$(SRCS:.cpp=.d) : %.d : %.cpp
$(CXX) $(CXXFLAGS) -o $@ $^ $(MKLLIB)
.PHONY : clean
clean:
rm -f $(LIBTREFIDE) $(OBJS) $(SRCS:.cpp=.d)
cd $(PROXTV); make clean;
cd $(GLMGEN); make clean;

It can be installed without error, but still crashed when run the demo.
I have tried everything, could you please help me out?

[davidgwyrick]

I'm running into this error as well. I followed all of the instructions in the Math Kernel Library Install and subsequent trefide build and install, but this error seems to persist.

Based on my copy-paste error googling, it seems as if this is a problem with other programs as well that use MKL. I tried most of the solutions people have proposed, but still get library linking errors. @RenieWell perhaps you can try explicitly loading the libraries like below. It seemed to work for some people.

$ LD_PRELOAD="/opt/intel/mkl/lib/intel64/libmkl_def.so:/opt/intel/mkl/lib/intel64/libmkl_avx.so" jupyter-lab --port 4242

symbol lookup error: /opt/intel/mkl/lib/intel64/libmkl_avx.so: undefined symbol: mkl_sparse_optimize_bsr_trsm_i8

[jtoloe]

I had the same issue. With preloading of libmkl_core.so and libmkl_sequential.so it worked and I can run the demo notebooks. I found this solution here.

LD_PRELOAD=$CONDA_PREFIX/libmkl_core.so:$CONDA_PREFIX/libmkl_sequential.so

[RenieWell]

I had the same issue. With preloading of libmkl_core.so and libmkl_sequential.so it worked and I can run the demo notebooks. I found this solution here.

LD_PRELOAD=$CONDA_PREFIX/libmkl_core.so:$CONDA_PREFIX/libmkl_sequential.so

Thanks jtoloe!
I solved it by using the new updated conda solution.

[guptarohit]

the following combination of packages (got from $ conda list) worked for me:

# Name                    # Version                   # Build
blas                      1.0                         mkl  
mkl                       2021.3.0           h06a4308_520  
mkl-service               2.4.0            py36h7f8727e_0  
mkl_fft                   1.3.0            py36h42c9631_2  
mkl_random                1.2.2            py36h51133e4_0 
numpy                     1.19.5                   pypi_0

following installs everything required:

conda install mkl_fft -y

☕ ✌️