Aesara fails to set correct blas ldflags in some windows installations

[lucianopaz]

Description of your problem or feature request

We’ve seen two independent reports of this happen in pymc. See here, here and here.

Please provide a minimal, self-contained, and reproducible example.

Both @opherdonchin and @pandrich said that they had installed pymc by doing

conda install pymc -c conda-forge

Into a new and clean environment. Then the following snippet failed

import aesara
from aesara import tensor as at

x = at.dvector(“x”)
b = at.dmatrix(“b”)
y = at.dot(b, x)
f = aesara.function([x, b], y)
f([1, 0], [[0.5, 0.5], [0.5, 0.5]])

Please provide the full traceback of any errors.

You can find the C code in this temporary file: C:\Users\OPHERL~1\AppData\Local\Temp\aesara_compilation_error_cqab8ef9
library mkl_rt is not found.
Traceback (most recent call last):
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\vm.py", line 1246, in make_all
node.op.make_thunk(node, storage_map, compute_map, [], impl=impl)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\op.py", line 131, in make_thunk
return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\op.py", line 96, in make_c_thunk
outputs = cl.make_thunk(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1202, in make_thunk
cthunk, module, in_storage, out_storage, error_storage = self.compile(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1122, in compile
thunk, module = self.cthunk_factory(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1647, in cthunk_factory
module = cache.module_from_key(key=key, lnk=self)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\cmodule.py", line 1229, in module_from_key
module = lnk.compile_cmodule(location)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1546, in compile_cmodule
module = c_compiler.compile_str(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\cmodule.py", line 2640, in compile_str
raise CompileError(
aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
"C:\miniconda3\envs\pymc_env\Library\mingw-w64\bin\g++.exe" -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=broadwell -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=generic -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -I"C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\numpy\core\include" -I"C:\miniconda3\envs\pymc_env_aesara\include" -I"C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\c_code" -L"C:\miniconda3\envs\pymc_env_aesara\libs" -L"C:\miniconda3\envs\pymc_env_aesara" -o "C:\Users\Opher local\AppData\Local\Aesara\compiledir_Windows-10-10.0.19043-SP0-Intel64_Family_6_Model_142_Stepping_12_GenuineIntel-3.10.6-64\tmp8du5yfuh\md2fc7729a4f0f5eb7d1fa94d7650200f3de6577f147c69a812b2b820fec26507.pyd" "C:\Users\Opher local\AppData\Local\Aesara\compiledir_Windows-10-10.0.19043-SP0-Intel64_Family_6_Model_142_Stepping_12_GenuineIntel-3.10.6-64\tmp8du5yfuh\mod.cpp" -lmkl_rt "C:\miniconda3\envs\pymc_env_aesara\python310.dll"
C:/miniconda3/envs/pymc_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmkl_rt
collect2.exe: error: ld returned 1 exit status

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\Opher\GitHub\bsf_donchin_jordan_2022\tests\test_aesara_flags.py", line 10, in 
f = aesara.function([x, b], y)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\compile\function_init_.py", line 317, in function
fn = pfunc(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\compile\function\pfunc.py", line 371, in pfunc
return orig_function(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\compile\function\types.py", line 1759, in orig_function
fn = m.create(defaults)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\compile\function\types.py", line 1652, in create
_fn, _i, _o = self.linker.make_thunk(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\basic.py", line 254, in make_thunk
return self.make_all(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\vm.py", line 1255, in make_all
raise_with_op(fgraph, node)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\utils.py", line 534, in raise_with_op
raise exc_value.with_traceback(exc_trace)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\vm.py", line 1246, in make_all
node.op.make_thunk(node, storage_map, compute_map, [], impl=impl)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\op.py", line 131, in make_thunk
return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\op.py", line 96, in make_c_thunk
outputs = cl.make_thunk(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1202, in make_thunk
cthunk, module, in_storage, out_storage, error_storage = self.compile(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1122, in compile
thunk, module = self.cthunk_factory(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1647, in cthunk_factory
module = cache.module_from_key(key=key, lnk=self)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\cmodule.py", line 1229, in module_from_key
module = lnk.compile_cmodule(location)
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\basic.py", line 1546, in compile_cmodule
module = c_compiler.compile_str(
File "C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\cmodule.py", line 2640, in compile_str
raise CompileError(
aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
"C:\miniconda3\envs\pymc_env\Library\mingw-w64\bin\g++.exe" -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=broadwell -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=generic -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -I"C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\numpy\core\include" -I"C:\miniconda3\envs\pymc_env_aesara\include" -I"C:\miniconda3\envs\pymc_env_aesara\lib\site-packages\aesara\link\c\c_code" -L"C:\miniconda3\envs\pymc_env_aesara\libs" -L"C:\miniconda3\envs\pymc_env_aesara" -o "C:\Users\Opher local\AppData\Local\Aesara\compiledir_Windows-10-10.0.19043-SP0-Intel64_Family_6_Model_142_Stepping_12_GenuineIntel-3.10.6-64\tmp8du5yfuh\md2fc7729a4f0f5eb7d1fa94d7650200f3de6577f147c69a812b2b820fec26507.pyd" "C:\Users\Opher local\AppData\Local\Aesara\compiledir_Windows-10-10.0.19043-SP0-Intel64_Family_6_Model_142_Stepping_12_GenuineIntel-3.10.6-64\tmp8du5yfuh\mod.cpp" -lmkl_rt "C:\miniconda3\envs\pymc_env_aesara\python310.dll"
C:/miniconda3/envs/pymc_env/Library/mingw-w64/bin/../lib/gcc/x86_64-w64-mingw32/5.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmkl_rt
collect2.exe: error: ld returned 1 exit status

Apply node that caused the error: CGemv{inplace}(AllocEmpty{dtype='float64'}.0, TensorConstant{1.0}, b, x, TensorConstant{0.0})
Toposort index: 2
Inputs types: [TensorType(float64, (None,)), TensorType(float64, ()), TensorType(float64, (None, None)), TensorType(float64, (None,)), TensorType(float64, ())]

HINT: Use a linker other than the C linker to print the inputs' shapes and strides.
HINT: Re-running with most Aesara optimizations disabled could provide a back-trace showing when this node was created. This can be done by setting the Aesara flag 'optimizer=fast_compile'. If that does not work, Aesara optimizations can be disabled with 'optimizer=None'.
HINT: Use the Aesara flag exception_verbosity=high for a debug print-out and storage map footprint of this Apply node.

Please provide any additional information below.

I’ll ask both @opherdonchin and @pandrich to post their aesara version numbers. You can see their aesara config in the linked issues/discussions, and note that the blas flags don’t include mkl, even though they are correctly installed by conda

Versions and main components

• Aesara version:
• Aesara config (python -c "import aesara; print(aesara.config)")
• Python version:
• Operating system:
• How did you install Aesara: (conda/pip)

[brandonwillard]

Both @opherdonchin and @pandrich said that they had installed pymc by doing

conda install pymc -c conda-forge

We need the MWE environment that causes the issue: i.e. the explicit package installations.

[brandonwillard]

Also, we already have an open Discussion for this. Why do we need a new issue?

[brandonwillard]

See #1239 (reply in thread). We can continue the discussion in #1239; closing this as a duplicate for now.