Issue installing Arpack with julia packaged for opensuse Tumbleweed #55

ericst · 2019-01-08T21:58:49Z

I have a problem when trying to install Arpac on opensuse Tumbleweed(x86_64). I installed Julia from the package manager.

Here is the output when I try to add Arpack:

(v1.0) pkg> add Arpack
 Resolving package versions...
 Installed Arpack ─ v0.3.0
  Updating `~/.julia/environments/v1.0/Project.toml`
  [7d9fca2a] + Arpack v0.3.0
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [7d9fca2a] + Arpack v0.3.0
  Building Arpack → `~/.julia/packages/Arpack/UiiMc/deps/build.log`
┌ Error: Error building `Arpack`: 
│ [ Info: Downloading https://github.com/JuliaLinearAlgebra/ArpackBuilder/releases/download/v3.5.0-3/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc8.tar.gz to /home/ericst/.julia/packages/Arpack/UiiMc/deps/usr/downloads/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc8.tar.gz...
│ ERROR: LoadError: LibraryProduct(nothing, ["libarpack"], :libarpack, "Prefix(/home/ericst/.julia/packages/Arpack/UiiMc/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] #write_deps_file#152(::Bool, ::Function, ::String, ::Array{LibraryProduct,1}) at /home/ericst/.julia/packages/BinaryProvider/4F5Hq/src/Products.jl:414
│  [3] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_file), ::String, ::Array{LibraryProduct,1}) at ./none:0
│  [4] top-level scope at none:0
│  [5] include at ./boot.jl:317 [inlined]
│  [6] include_relative(::Module, ::String) at ./loading.jl:1044
│  [7] include(::Module, ::String) at ./sysimg.jl:29
│  [8] include(::String) at ./client.jl:392
│  [9] top-level scope at none:0
│ in expression starting at /home/ericst/.julia/packages/Arpack/UiiMc/deps/build.jl:74
[20:40:53] ######################################################################## 100.0% 
└ @ Pkg.Operations /home/abuild/rpmbuild/BUILD/julia-1.0.3/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1097

And here is the output of versioninfo():

julia> versioninfo()
Julia Version 1.0.3
Commit 04330c0378 (2018-12-16 21:23 UTC)
Platform Info:
  OS: Linux (x86_64-suse-linux)
  CPU: Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, broadwell)

After that I tried by downloading the binaries from julialang.org and using them instead and I haven't had any problems with that. Here is then the versioninfo() output for this:

julia> versioninfo()
Julia Version 1.0.3
Commit 099e826241 (2018-12-18 01:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, broadwell)

The text was updated successfully, but these errors were encountered:

andreasnoack · 2019-01-08T22:22:53Z

@staticfloat Wasn't this supposed to work now?

staticfloat · 2019-01-09T08:06:28Z

From-source builds and binary builds work; this sounds like a distribution has packaged something that doesn't match our defaults. @ericst can you tell me what the name of your BLAS library is? You can do that via the following:

julia> using Libdl

julia> filter(x -> occursin("blas", x), dllist())
1-element Array{AbstractString,1}:
 "/home/staticfloat/dist/julia-1.0.3/bin/../lib/julia/libopenblas64_.so"

rlefmann · 2019-01-10T14:32:29Z

I have the same problem. I am on Manjaro and Julia is installed from the package manager. I am using the Julia commit that works for @ericst

julia> versioninfo()
Julia Version 1.0.3
Commit 099e826241* (2018-12-18 01:34 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i3-3227U CPU @ 1.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, ivybridge)

BLAS library:

julia> using Libdl
julia> filter(x -> occursin("blas", x), dllist())
1-element Array{AbstractString,1}:
 "/usr/bin/../lib/julia/libblas.so"

Luckily for us, we don't have the problem of distribution packagers changing BLAS library names on platforms like Windows where this solutino wouldn't work. Eventually, this will most likely be obsoleted by the linker hacks that Keno and I will write to fix linking multiple disparate folders together.

staticfloat · 2019-01-10T22:28:53Z

Can those with these problems please run pkg> dev Arpack, then run:

cd ~/.julia/dev/Arpack/deps
git fetch -a
git checkout sf/distribution_compatibility
rm -rf usr
julia --color=yes -i build.jl

And show me what the output looks like? If you have this problem, I expect to see something like the following:

[ Info: Downloading https://github.com/JuliaLinearAlgebra/ArpackBuilder/releases/download/v3.5.0-3/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc7.tar.gz to /home/staticfloat/.julia/dev/Arpack/deps/usr/downloads/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc7.tar.gz...
[17:24:07] ######################################################################## 100.0%
┌ Warning: This Julia installation uses a non-default BLAS library name ("libblas.so" instead of "libopenblas64_.so.0")!  This is likely due to using a distribution package, and can cause problems when installing binary dependencies such as Arpack.jl. We will attempt to automatically map the distribution-provided BLAS library to such that libarpack can find it, but this may not work.  If issues with Arpack.jl persist, we recommend using the Julia binaries available for download from the official website at https://julialang.org/downloads.html
└ @ Main ~/.julia/dev/Arpack/deps/build.jl:127
┌ Warning: Attempting to symlink "/home/staticfloat/dist/julia-1.0.3/lib/libopenblas64_.so.0" => "/lib/x86_64-linux-gnu/libblas.so"; if this fails, try running it manually with super user permissions via: sudod ln -s /lib/x86_64-linux-gnu/libblas.so.6 /home/staticfloat/dist/julia-1.0.3/lib/libopenblas64_.so.0

If it doesn't work (e.g. you get a permissions error) then you need to do it with sudo privileges, the warning message above will print out what you need to do, so just copy-paste that. And note that the simplest solution is of course to use the official binaries which you can download from here.

sostock · 2019-01-11T20:52:52Z

For me, the problem exists on Arch Linux x86_64 using the package julia 2:1.0.3-2 from the community repo. This is the output of sudo julia --color=yes -i build.jl:

[ Info: Precompiling BinaryProvider [b99e7846-7c00-51b0-8f62-c81ae34c0232]
[ Info: Downloading https://github.com/JuliaLinearAlgebra/ArpackBuilder/releases/download/v3.5.0-3/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc8.tar.gz to /home/sebastian/.julia/dev/Arpack/deps/usr/downloads/Arpack.v3.5.0-3.x86_64-linux-gnu-gcc8.tar.gz...
[20:59:35] ######################################################################## 100,0%
┌ Warning: This Julia installation uses a non-default BLAS library name ("libblas.so" instead of "libopenblas64_.so.0")!  This is likely due to using a distribution package, and can cause problems when installing binary dependencies such as Arpack.jl. We will attempt to automatically map the distribution-provided BLAS library to such that libarpack can find it, but this may not work.  If issues with Arpack.jl persist, we recommend using the Julia binaries available for download from the official website at https://julialang.org/downloads/
└ @ Main /home/sebastian/.julia/dev/Arpack/deps/build.jl:127
┌ Warning: Attempting to symlink "/usr/lib/libopenblas64_.so.0" => "/usr/bin/../lib/julia/libblas.so"; if this fails, try running it manually with super user permissions via: sudod ln -s /usr/bin/../lib/julia/libblas.so /usr/lib/libopenblas64_.so.0
└ @ Main /home/sebastian/.julia/dev/Arpack/deps/build.jl:138

With the package julia-bin 1.0.3-1 from the Arch User Repository, I can install and use Arpack.jl without problems.

However, on my machine, both julia from the community repo and julia-bin from the AUR fail their test suite (Base.runtests) with a couple of mkdir: Permission denied errors (julia-bin throws fewer errors than julia, but still fails). The official binaries from https://julialang.org/downloads/ succeed.

staticfloat · 2019-01-11T21:53:20Z

@sostock; that looks like it worked; great. (Although since you ran things as root, it's possible that files within your ~/.julia folder are now owned by root; I suggest not running julia as root and only running the ln command via sudo). Does your julia from the community repo work with Arpack.jl now?

However, on my machine, both julia from the community repo and julia-bin from the AUR fail their test suite

If this bothers you, I would suggest opening issues about that on the community repo and AUR.

sostock · 2019-01-12T14:35:15Z

@staticfloat Actually, creating the symlink did not help with the community repo Julia. I can successfully build Arpack, but test Arpack results in an error:

/usr/bin/julia: symbol lookup error: /home/sebastian/.julia/dev/Arpack/deps/usr/lib/libarpack.so: undefined symbol: dlamch_64_

In the julia package from the community repo, /usr/lib/julia/libblas.so is a symlink to /usr/lib/libopenblas.so.3, which is a symlink to /usr/lib/libopenblasp-r0.3.5.so.

For now, I will just use julia-bin from the AUR since it works without problems.

staticfloat · 2019-01-12T18:35:17Z

Ah; yeah. If they aren't using an ILP64 BLAS, then there's really nothing we can do. My suggestion is to use Julia distributions that don't change the build configuration to incompatible settings.

ericst · 2019-01-14T22:16:22Z

Well yeah. Then using the download from julialang.org is good. Thank you very much.

jg110 · 2019-01-16T15:18:43Z

Can those with these problems please run pkg> dev Arpack, then run:
cd ~/.julia/dev/Arpack/deps
git fetch -a
git checkout sf/distribution_compatibility
rm -rf usr
julia --color=yes -i build.jl

This worked for me on Fedora 29. The last line initially failed, but after creating the symlink with sudo, Arpack builds fine.

venkatachalapathy · 2019-06-12T14:00:24Z

The fix suggested by @staticfloat worked for my old Ubuntu 14.04 machine. I had no further issues. Thanks!

j-fu · 2019-09-09T12:38:11Z

Hi, the issue still persists (openSUSE 15.0 and 15.1, julia 1.1 installed via system repos).
The fix suggested by @staticfloat requires superuser intervention and thus is not feasible for users on a server installation. I also think that it is not always the simplest way to download julia from the julia website. The possibility of maintenance via system repositories by itself is great, and in this respect it would be a pity if Julia would behave differently compared to octave or scilab.

That said, I see the problem behind the issue.
But: First of all it is surprising to me that the compilation process does not try to use the libopenblas with anyway comes around with julia (I think via suitesparse):

julia> using Libdl
julia> filter(x -> occursin("blas", x), dllist())
2-element Array{AbstractString,1}:
 "/usr/bin/../lib64/julia/libopenblas.so"
 "/usr/lib64/libblas.so.3"

/usr/lib64/julia/libopenblas.so in fact belongs to the openSUSE Julia package from the science repo.

Second, to me it not obvious why libopenblas64_.so is a good default for the library name which can be expected to be maintained by a package manager. From our own experience with compiling code, libopenblas and libblas are the two most common names, and historical names (libcomplib.sgimath, libsunmath, libcxml ) cease to be relevant.
UPDATE: Ok, after diving further into the build system structure I start seeing the point. While I still think that a distribution by system vendors has its merits, it is hard to keep everything consistent if Julia is forced to work with system provided libraries. So it seems it is a packaging error and I think about filing an issue there. It seems that they need to fix the BLAS library and the USE_BLAS64 flag.

j-fu · 2019-10-27T16:07:50Z

... me again. Had to switch to openSUSE Tumbleweed due to a new laptop with wifi6 which needs kernel version 5.
Meanwhile, I found out about #69 which suggests to just copy a compiled version to the julia directory (Julia install via system repos). This worked.

danmrc · 2019-11-22T01:15:49Z

Even less orthodoxy solution, and seems to work fine here (EndeavourOS, using julia from arch/comunity) (but failed during tests):
0. Install Arpack using pacman

Create a deps.jl file anywhere with only the following:

const libarpack = joinpath(dirname(@__FILE__), "usr/include/arpack/arpack.hpp")

Put it on the build folder of Arpack

This might be a bad idea?

ViralBShah · 2019-12-03T03:54:55Z

A lot of this should be addressed by recent improvements to BinaryBuilder. I suggest opening a new issue if still necessary.

ericst mentioned this issue Jan 8, 2019

Build error with 1.0.3 on AARCH64 #54

Closed

ViralBShah closed this as completed Dec 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue installing Arpack with julia packaged for opensuse Tumbleweed #55

Issue installing Arpack with julia packaged for opensuse Tumbleweed #55

ericst commented Jan 8, 2019

andreasnoack commented Jan 8, 2019

staticfloat commented Jan 9, 2019

rlefmann commented Jan 10, 2019 •

edited

Loading

staticfloat commented Jan 10, 2019 •

edited

Loading

sostock commented Jan 11, 2019

staticfloat commented Jan 11, 2019

sostock commented Jan 12, 2019

staticfloat commented Jan 12, 2019

ericst commented Jan 14, 2019

jg110 commented Jan 16, 2019

venkatachalapathy commented Jun 12, 2019

j-fu commented Sep 9, 2019 •

edited

Loading

j-fu commented Oct 27, 2019

danmrc commented Nov 22, 2019

ViralBShah commented Dec 3, 2019

Issue installing Arpack with julia packaged for opensuse Tumbleweed #55

Issue installing Arpack with julia packaged for opensuse Tumbleweed #55

Comments

ericst commented Jan 8, 2019

andreasnoack commented Jan 8, 2019

staticfloat commented Jan 9, 2019

rlefmann commented Jan 10, 2019 • edited Loading

staticfloat commented Jan 10, 2019 • edited Loading

sostock commented Jan 11, 2019

staticfloat commented Jan 11, 2019

sostock commented Jan 12, 2019

staticfloat commented Jan 12, 2019

ericst commented Jan 14, 2019

jg110 commented Jan 16, 2019

venkatachalapathy commented Jun 12, 2019

j-fu commented Sep 9, 2019 • edited Loading

j-fu commented Oct 27, 2019

danmrc commented Nov 22, 2019

ViralBShah commented Dec 3, 2019

rlefmann commented Jan 10, 2019 •

edited

Loading

staticfloat commented Jan 10, 2019 •

edited

Loading

j-fu commented Sep 9, 2019 •

edited

Loading