Using Arpack with MKL

[dmbates]

julia> versioninfo()
Julia Version 0.7.0-beta.65
Commit e2de9b3d2e* (2018-06-28 03:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)

(v0.7) pkg> st -m
    Status `~/.julia/environments/v0.7/Manifest.toml`
  [7d9fca2a] Arpack v0.1.1
  [6e4b80f9] BenchmarkTools v0.3.1
  [b99e7846] BinaryProvider v0.3.2
  [324d7699] CategoricalArrays v0.3.10
  [944b1d66] CodecZlib v0.4.3
  [34da2185] Compat v0.69.0
  [a93c6f00] DataFrames v0.11.6
  [9a8bc11e] DataStreams v0.3.6
  [864edb3b] DataStructures v0.8.3
  [ffbed154] DocStringExtensions v0.4.4
  [e30172f5] Documenter v0.18.0
  [682c06a0] JSON v0.17.2
  [e1d29d7a] Missings v0.2.10
  [73a701b4] NamedTuples v4.0.2
  [4d1e1d77] Nullables v0.0.5
  [90014a1f] PDMats v0.9.4
  [189a3867] Reexport v0.1.0
  [a2af1166] SortingAlgorithms v0.2.1
  [2913bbd2] StatsBase v0.23.1
  [3bb67fe8] TranscodingStreams v0.5.2
  [ea10d353] WeakRefStrings v0.4.7
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8bb1440f] DelimitedFiles 
  [8ba89e20] Distributed 
  [9fa8497b] Future 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [a63ad114] Mmap 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [1a1011a3] SharedArrays 
  [6462fe0b] Sockets 
  [2f01184e] SparseArrays 
  [4607b0f0] SuiteSparse 
  [8dfed614] Test 
  [cf7118a7] UUIDs 
  [4ec0a83e] Unicode 

(v0.7) pkg> build Arpack
  Building Arpack → `~/.julia/packages/Arpack/8rPU/deps/build.log`
┌ Error: Error building `Arpack`: 
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = macro expansion at OutputCollector.jl:63 [inlined]
│ └ @ Core OutputCollector.jl:63
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = wait(::OutputCollector) at OutputCollector.jl:158
│ └ @ BinaryProvider OutputCollector.jl:158
│ ┌ Warning: `wait(t::Task)` is deprecated, use `fetch(t)` instead.
│ │   caller = wait(::OutputCollector) at OutputCollector.jl:159
│ └ @ BinaryProvider OutputCollector.jl:159
│ ERROR: LoadError: LibraryProduct(nothing, ["libarpack"], :libarpack, "Prefix(/home/bates/.julia/packages/Arpack/8rPU/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error at ./error.jl:33 [inlined]
│  [2] #write_deps_file#134(::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at /home/bates/.julia/packages/BinaryProvider/2Hlv/src/Products.jl:389
│  [3] write_deps_file(::String, ::Array{LibraryProduct,1}) at /home/bates/.julia/packages/BinaryProvider/2Hlv/src/Products.jl:376
│  [4] top-level scope at none:0
│  [5] include at ./boot.jl:317 [inlined]
│  [6] include_relative(::Module, ::String) at ./loading.jl:1075
│  [7] include(::Module, ::String) at ./sysimg.jl:29
│  [8] include(::String) at ./client.jl:393
│  [9] top-level scope at none:0
│ in expression starting at /home/bates/.julia/packages/Arpack/8rPU/deps/build.jl:40
└ @ Pkg.Operations Operations.jl:973

[andreasnoack]

This has been reported a couple of times. It only happens when Julia has been built from source but we are we are trying to figure out how to fix it. Meanwhile, you can either use the official binaries or try building OpenBLAS with gfortran-7, i.e. something like

make -C deps distclean-openblas
make -C deps -j install-openblas FC=gfortran-7

and then set LD_LIBRARY_PATH such that libgfortran.4 is visible.

[dmbates]

I am building with MKL, not OpenBLAS. My Make.user file is

USE_INTEL_MKL = 1
FFLAGS = -ff2c

Is there a workaround in this case?

[andreasnoack]

Not yet, unfortunately. I suspect we'd need to do separate binaries for MKL. cc @staticfloat

[vtjnash]

Essentially the same issue as #5

[joaquimg]

I have just noted https://github.com/JuliaPackaging/Yggdrasil/tree/master/I/IpoptMKL besides https://github.com/JuliaPackaging/Yggdrasil/tree/master/I/Ipopt
The idea would be to do the same for Arpack?

[ViralBShah]

In v1.7, we should link Arpack_jll against LBT, which will allow it to use MKL. This isn't too far away, and we should probably wait until 1.7 releases before making this change.

[ghost]

In v1.7, we should link Arpack_jll against LBT, which will allow it to use MKL. This isn't too far away, and we should probably wait until 1.7 releases before making this change.

When can this update be expected, and is there a temporary fix in the meantime?

[ViralBShah]

The right way to do this would be right after 1.7 releases.

[ViralBShah]

I'll give it a shot and get some of the builds going to start with.

[ghost]

I'll give it a shot and get some of the builds going to start with.

Any progress on fixing this?

[ViralBShah]

It's easiest for me to do this as 1.7 and later. Trying to make it work in 1.6 and earlier, is substantial work that I am also not sure about.

[ParadaCarleton]

It's easiest for me to do this as 1.7 and later. Trying to make it work in 1.6 and earlier, is substantial work that I am also not sure about.

Is it possible to use Arpack+MKL already in the 1.7 dev build, or would I have to wait until the full stable release?

[ViralBShah]

No it isn't possible, since ARPACK is linked against openblas right now. We'll switch it to the new mechanism either close to or soon after 1.7.

[ViralBShah]

With JuliaPackaging/Yggdrasil#3699 now closed, it should be possible to link Arpack_jll against libblastrampoline_jll (instead of openblas), which will then allow for switching to MKL in Julia 1.7 and higher.

[ViralBShah]

Doing using MKL before using Arpack should make Arpack use MKL now (Arpack is linked against LBT). Naturally, this hasn't really been tested a whole lot, but it seems to pass the tests.