Linking error on Windows 11 using Intel MKL as static library

[cgranade]

When compiling a small crate against ndarray-linalg using the intel-mkl-static feature, linking errors result depending on the value of lib.crate-type in Cargo.toml.

Example crate

// src/lib.rs
use num_complex::Complex64 as c64;
use ndarray::{Array2, ArrayView2};

pub trait HasDagger {
    type Output;

    fn dag(&self) -> Self::Output;
}

impl HasDagger for Array2<c64> {
    type Output = Self;

    fn dag(&self) -> Self {
        self.t().map(|element| element.conj())
    }
}

impl HasDagger for ArrayView2<'_, c64> {
    type Output = Array2<c64>;

    fn dag(&self) -> Self::Output {
        self.t().map(|element| element.conj())
    }
}

pub trait ConjBy {
    fn conjugate_by(&self, op: &ArrayView2<c64>) -> Self;
}

impl ConjBy for Array2<c64> {
    fn conjugate_by(&self, op: &ArrayView2<c64>) -> Self {
        op.dot(self).dot(&op.dag())
    }
}

[package]
name = "blas-rs"
version = "0.0.1"
edition = '2018'

[lib]
crate-type = [
    'rlib',
    'staticlib',
    'cdylib',
]

[dependencies]
anyhow = '<1.0.49'

[dependencies.ndarray]
version = '0.15.4'
features = ['serde']

[dependencies.num-complex]
version = '0.4'
features = ['serde']

[dependencies.serde]
version = '1.0'
features = ['derive']

[dependencies.ndarray-linalg]
version = '0.14.1'
features = ['intel-mkl-static']

Example error:

$ cargo build
➜  cargo build
   Compiling blas-rs v0.0.1 (C:\Users\cgran\source\scratch\blas-rs)
warning: Error finalizing incremental compilation session directory `\\?\C:\Users\cgran\source\scratch\blas-rs\target\debug\incremental\blas_rs-204zbe9n3gp6o\s-g6at8u30m7-1gpl7ds-working`: Access is denied. (os error 5)

error: linking with `link.exe` failed: exit code: 1120
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX64\\x64\\link.exe"
  = note:    Creating library C:\...\target\debug\deps\blas_rs.dll.lib and object C:\...\target\debug\deps\blas_rs.dll.exp
          blas_rs.pi0kkv54xyu470e.rcgu.o : error LNK2019: unresolved external symbol cblas_sgemm referenced in function _ZN7ndarray6linalg11impl_linalg12mat_mul_impl17h80b4dcd4e156b29cE
          blas_rs.pi0kkv54xyu470e.rcgu.o : error LNK2019: unresolved external symbol cblas_dgemm referenced in function _ZN7ndarray6linalg11impl_linalg12mat_mul_impl17h80b4dcd4e156b29cE
          blas_rs.pi0kkv54xyu470e.rcgu.o : error LNK2019: unresolved external symbol cblas_cgemm referenced in function _ZN7ndarray6linalg11impl_linalg12mat_mul_impl17h80b4dcd4e156b29cE
          blas_rs.pi0kkv54xyu470e.rcgu.o : error LNK2019: unresolved external symbol cblas_zgemm referenced in function _ZN7ndarray6linalg11impl_linalg12mat_mul_impl17h80b4dcd4e156b29cE
          C:\Users\cgran\source\scratch\blas-rs\target\debug\deps\blas_rs.dll : fatal error LNK1120: 4 unresolved externals


warning: `blas-rs` (lib) generated 1 warning
error: could not compile `blas-rs` due to previous error; 1 warning emitted

Running cargo build works for some settings of lib.crate-type, however:

• ["rlib", "staticlib", "cdylib"]: fails
• ["staticlib", "cdylib"]: works
• ["rlib", "staticlib"]: works
• ["rlib"]: works
• ["staticlib"]: works
• ["rlib", "cdylib"]: fails
• ["cdylib"]: works

Version info

➜ cargo --version
cargo 1.57.0 (b2e52d7ca 2021-10-21)

➜ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\...\.rustup

installed toolchains
--------------------

stable-x86_64-pc-windows-msvc
nightly-x86_64-pc-windows-msvc

installed targets for active toolchain
--------------------------------------

x86_64-apple-darwin
x86_64-pc-windows-gnu
x86_64-pc-windows-msvc
x86_64-unknown-linux-gnu

active toolchain
----------------

stable-x86_64-pc-windows-msvc (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

[cgranade]

As a follow-up, this seems possibly consistent with LTO bugs in rustc itself: rust-lang/rust#51009

[jturner314]

I'm not able to reproduce this error on Arch Linux with

% rustc --version
rustc 1.58.1 (db9d1b20b 2022-01-20)
% cargo --version
cargo 1.58.0 (f01b232bc 2022-01-19)

That warning before the error message is strange:

warning: Error finalizing incremental compilation session directory \\?\C:\Users\cgran\source\scratch\blas-rs\target\debug\incremental\blas_rs-204zbe9n3gp6o\s-g6at8u30m7-1gpl7ds-working: Access is denied. (os error 5)

Do you have any ideas regarding that message?

Do you still get an error if you run cargo clean before cargo build?

[cgranade]

Apologies for missing your reply! I did still get the issue on Windows 11 even when running cargo clean before cargo build. For the warning message, I sometimes have seen that due to the rust-analyzer extension for VS Code locking cache folders; I had missed that that happened in this case, let me try and rerun from the command-line with rust-analyzer closed and see if I can reproduce. Thank you!

[cgranade]

Rerunning at the command-line with rust-analyzer disabled, I reproduce the above on Windows 11:

➜  rustc --version
rustc 1.57.0 (f1edd0429 2021-11-29)

➜  cargo --version
cargo 1.57.0 (b2e52d7ca 2021-10-21)

The error persists after running rustup update:

➜  rustc --version
rustc 1.58.1 (db9d1b20b 2022-01-20)

➜  cargo --version
cargo 1.58.0 (f01b232bc 2022-01-19)

Running in a fresh Docker container from the rust image, the build completes successfully and produces all three artifacts in target/debug, such that the issue looks to possibly be Windows-specific:

# ls target/debug/
build  deps  examples  incremental  libblas_rs.a  libblas_rs.d  libblas_rs.rlib  libblas_rs.so
# cat /etc/issue
Debian GNU/Linux 11 \n \l

[jturner314]

I'm not sure I'll be able to help much with this; my time is pretty limited, and I don't have a Windows machine set up for Rust development. Some suggestions:

• Try building a crate with just lapack and intel-mkl-src dependencies (with the appropriate features enabled), to see if the issue is with ndarray-linalg or the dependencies. (More info here.) If the issue is with the dependencies, then you could try creating an issue on the lapack repository to see if they could provide more help.

• You could try the Rust users forum or chat platforms.

• Possible workarounds:

  • Try creating a second crate which just wraps the first and exports the necessary functionality. (The inner crate would be an rlib, and the outer crate would be a cdylib.) This may work around whatever is causing the issue, since it occurs only when trying to build a single crate as both an rlib and a cdylib.

  • Cross-compile for Windows using Linux.

  • Use a different backend, e.g. OpenBLAS.

[prokie]

I see this issue on Windows Server but not on my windows 10 and windows 11 machine. I cant figure out why it does not work on the Windows server machine.