Installation error when Julia is built from source #5
Comments
|
Might be dup of #3. Did you build from source? What do you get from |
|
I encountered the same problem on macOS, 0.7.0-beta.0. Do we need to wait for the official binaries of Julia 0.7.0-beta? |
Either that or build your OpenBLAS with GCC 7. |
|
Thank you. I will try. |
|
@andreasnoack this was with a build from source. Thanks to the pointer to Viral's issue – I didn't see it since it was closed. |
|
@yurivish You'll need to adjust the path to match the |
|
|
FYI, Julia 0.7-beta binary for macOS is now available (https://julialang.org/downloads/), which solves the problem. |
|
@blakejohnson Things are more wrong than first anticipated. So the system |
|
Another possible workaround is to copy (or maybe hard link) libgfortran.so.4 from another Julia package (e.g. SpecialFunctions) into |
|
Just for completeness (coming from #9), this obviously also occurs on Linux when not building with GCC 7. Doesn't seem easy to work around, with eg. Arch Linux' |
andreasnoack
changed the title
|
Until we fix this, could we throw a more informative error? |
|
The new tag of Arpack.jl now fails to compile on Julia binaries as well: julia> using Arpack
[ Info: Precompiling module Arpack
ERROR: LoadError: No deps.jl file could be found. Please try running Pkg.build("Arpack").
Currently, the build command might fail when Julia has been built from source
and the recommendation is to use the official binaries from julialang.org.
For more info see https://github.com/JuliaLinearAlgebra/Arpack.jl/issues/5.Julia version: julia> versioninfo()
Julia Version 0.7.0-beta.182
Commit feaee9ebbc (2018-07-06 12:39 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.0 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 2 |
|
Did you actually run |
|
I did, there is no error message, and the build.log is empty. |
|
It the test for the |
|
This may be stating the obvious, but the problem still exists with 1.4 on a Julia built from source (which is the only way I can do it right now). Is there a workaround for those of us in this unfortunate position? |
|
You can try downloading the official Julia binaries, extracting the |
|
Thank you for the followup.
Unfortunately I won't be able to do that due to environmental restrictions here. I know people are working on this, but to set expectations, is there an approximate timeframe? The lack of a working |
It will definitely be more than a few days; it's very complex to create infrastructure to support every reasonable toolchain (rather than a single reasonable toolchain, which is what we do right now); think single-digit months rather than single-digit days. If it is the official Julia distribution itself that is the problem, you don't have to get If none of those solutions will work in your environment, then I'm afraid you're just going to have to wait. |
|
Thanks. That's disappointing but I'm not in a position to help other than state my use case.
I'm curious about this. I wouldn't think that downloading and building the Julia source code would be so far outside the norm that it's not part of the testing strategy. Are people doing something else? Because the upshot is that over 600 packages are using Arpack, and none of them will work on source-built installations. (This is not really so much a complaint as a question of whether our environment is really that strange.) |
|
Would it be feasible to support building Arpack from source as a generic fallback instead of downloading the binary? |
Building Julia from source works just fine with these binaries, as long as you use a compiler version that is compatible with the binaries (e.g. GCC 7.x) or alternatively, are able to install the compiler support libraries ( This transition pain is considered worthwhile because the alternatives (using distro-provided package managers, building from source on user's machines, etc...) run into new issues constantly. It is a very large, ongoing, maintenance burden for us to constantly fix build scripts, debug user's broken toolchains, etc... It is much simpler for everyone if the binaries are built once, and simply distributed to users in the format they were going to compile down to anyway. This of course raises the compiler compatibility issues, but I will also point out those compiler compatibility issues already existed in the "bad old days"; if you were using the official Julia binaries (as most users are) and you have to compile Arpack, you can run into similar problems. Not the exact same ones (where libraries are missing) but more subtle ones, where symbol duplication means that when Arpack tries to call a function whose functionality has changed between two compiler ABI versions, it just functions incorrectly/crashes.
Ah, that gives me another idea; Seth, for your particular use case, I'm sure you can manually compile Arpack and just insert it directly into # First, clone arpack-ng and checkout a known good gitsha
git clone https://github.com/opencollab/arpack-ng.git arpack-ng
cd arpack-ng
git checkout b095052372aa95d4281a645ee1e367c28255c947
# We'll build inside the `build` directory
mkdir build; cd build/
# We're going to install to ~/.julia/dev/Arpack/deps/usr. Change this if you have a different target installation location
prefix=$(echo ~/.julia/dev/Arpack/deps/usr)
# We need to link against OpenBLAS; get that from Julia: (Note; this only works for Julia 0.7)
openblas_dir=$(julia -e 'using Libdl; println(abspath(dirname(Libdl.dlpath(Libdl.dlopen(Base.libblas_name)))))')
# Use cmake to configure, with a huge number of symbol renames for the fortran code.
cmake .. -DCMAKE_INSTALL_PREFIX=$prefix -DBUILD_SHARED_LIBS=ON -DBLAS_LIBRARIES="-L$openblas_dir -lopenblas64_" -DLAPACK_LIBRARIES="-L$prefix/lib -lopenblas64_" -DCMAKE_Fortran_FLAGS="-O2 -fPIC -ffixed-line-length-none -cpp -fdefault-integer-8 -Dsaxpy=saxpy_64 -Ddaxpy=daxpy_64 -Dscopy=scopy_64 -Ddcopy=dcopy_64 -Dsgemv=sgemv_64 -Ddgemv=dgemv_64 -Dsgeqr2=sgeqr2_64 -Ddgeqr2=dgeqr2_64 -Dslacpy=slacpy_64 -Ddlacpy=dlacpy_64 -Dslahqr=slahqr_64 -Ddlahqr=dlahqr_64 -Dslanhs=slanhs_64 -Ddlanhs=dlanhs_64 -Dslarnv=slarnv_64 -Ddlarnv=dlarnv_64 -Dslartg=slartg_64 -Ddlartg=dlartg_64 -Dslascl=slascl_64 -Ddlascl=dlascl_64 -Dslaset=slaset_64 -Ddlaset=dlaset_64 -Dsscal=sscal_64 -Ddscal=dscal_64 -Dstrevc=strevc_64 -Ddtrevc=dtrevc_64 -Dstrmm=strmm_64 -Ddtrmm=dtrmm_64 -Dstrsen=strsen_64 -Ddtrsen=dtrsen_64 -Dsgbmv=sgbmv_64 -Ddgbmv=dgbmv_64 -Dsgbtrf=sgbtrf_64 -Ddgbtrf=dgbtrf_64 -Dsgbtrs=sgbtrs_64 -Ddgbtrs=dgbtrs_64 -Dsgttrf=sgttrf_64 -Ddgttrf=dgttrf_64 -Dsgttrs=sgttrs_64 -Ddgttrs=dgttrs_64 -Dspttrf=spttrf_64 -Ddpttrf=dpttrf_64 -Dspttrs=spttrs_64 -Ddpttrs=dpttrs_64 -Dsdot=sdot_64 -Dddot=ddot_64 -Dsger=sger_64 -Ddger=dger_64 -Dslabad=slabad_64 -Ddlabad=dlabad_64 -Dslaev2=slaev2_64 -Ddlaev2=dlaev2_64 -Dslamch=slamch_64 -Ddlamch=dlamch_64 -Dslanst=slanst_64 -Ddlanst=dlanst_64 -Dslanv2=slanv2_64 -Ddlanv2=dlanv2_64 -Dslapy2=slapy2_64 -Ddlapy2=dlapy2_64 -Dslarf=slarf_64 -Ddlarf=dlarf_64 -Dslarfg=slarfg_64 -Ddlarfg=dlarfg_64 -Dslasr=slasr_64 -Ddlasr=dlasr_64 -Dsnrm2=snrm2_64 -Ddnrm2=dnrm2_64 -Dsorm2r=sorm2r_64 -Ddorm2r=dorm2r_64 -Dsrot=srot_64 -Ddrot=drot_64 -Dssteqr=ssteqr_64 -Ddsteqr=dsteqr_64 -Dsswap=sswap_64 -Ddswap=dswap_64 -Dcaxpy=caxpy_64 -Dzaxpy=zaxpy_64 -Dccopy=ccopy_64 -Dzcopy=zcopy_64 -Dcgemv=cgemv_64 -Dzgemv=zgemv_64 -Dcgeqr2=cgeqr2_64 -Dzgeqr2=zgeqr2_64 -Dclacpy=clacpy_64 -Dzlacpy=zlacpy_64 -Dclahqr=clahqr_64 -Dzlahqr=zlahqr_64 -Dclanhs=clanhs_64 -Dzlanhs=zlanhs_64 -Dclarnv=clarnv_64 -Dzlarnv=zlarnv_64 -Dclartg=clartg_64 -Dzlartg=zlartg_64 -Dclascl=clascl_64 -Dzlascl=zlascl_64 -Dclaset=claset_64 -Dzlaset=zlaset_64 -Dcscal=cscal_64 -Dzscal=zscal_64 -Dctrevc=ctrevc_64 -Dztrevc=ztrevc_64 -Dctrmm=ctrmm_64 -Dztrmm=ztrmm_64 -Dctrsen=ctrsen_64 -Dztrsen=ztrsen_64 -Dcgbmv=cgbmv_64 -Dzgbmv=zgbmv_64 -Dcgbtrf=cgbtrf_64 -Dzgbtrf=zgbtrf_64 -Dcgbtrs=cgbtrs_64 -Dzgbtrs=zgbtrs_64 -Dcgttrf=cgttrf_64 -Dzgttrf=zgttrf_64 -Dcgttrs=cgttrs_64 -Dzgttrs=zgttrs_64 -Dcpttrf=cpttrf_64 -Dzpttrf=zpttrf_64 -Dcpttrs=cpttrs_64 -Dzpttrs=zpttrs_64 -Dcdotc=cdotc_64 -Dzdotc=zdotc_64 -Dcgeru=cgeru_64 -Dzgeru=zgeru_64 -Dcunm2r=cunm2r_64 -Dzunm2r=zunm2r_64 -DSCOPY=SCOPY_64 -DDCOPY=DCOPY_64 -DSLABAD=SLABAD_64 -DDLABAD=DLABAD_64 -DSLAMCH=SLAMCH_64 -DDLAMCH=DLAMCH_64 -DSLANHS=SLANHS_64 -DDLANHS=DLANHS_64 -DSLANV2=SLANV2_64 -DDLANV2=DLANV2_64 -DSLARFG=SLARFG_64 -DDLARFG=DLARFG_64 -DSROT=SROT_64 -DDROT=DROT_64 -DSGEMV=SGEMV_64 -DDGEMV=DGEMV_64 -Dscnrm2=scnrm2_64 -Ddznrm2=dznrm2_64 -Dcsscal=csscal_64 -Dzdscal=zdscal_64"
# Do the make install
make installOnce you've verified that the necessary files have been dumped into your Arpack's ## This file autogenerated by BinaryProvider.write_deps_file().
## Do not edit.
##
## Include this file within your main top-level source, and call
## `check_deps()` from within your module's `__init__()` method
const Arpack = joinpath(dirname(@__FILE__), "usr/lib64/libarpack.so")
function check_deps()
global Arpack
if !isfile(Arpack)
error("$(Arpack) does not exist, Please re-run Pkg.build(\"Arpack\"), and restart Julia.")
end
if Libdl.dlopen_e(Arpack) == C_NULL
error("$(Arpack) cannot be opened, Please re-run Pkg.build(\"Arpack\"), and restart Julia.")
end
endNote that on my machine, the |
|
One concrete issue is that Julia has certain requirements for how BLAS and Arpack are built and those requirements are often not satisfied by system libraries. We could configure Julia to work around the inconsistencies of system libraries on various systems but that just forces those inconsistencies onto our users, which we do not want to do. We have chosen instead of ship our own versions of such libraries so that we provide a maximally consistent and portable experience to our users—and they do appreciate it. Because of this philosophy, Julia code runs very much the same everywhere. The Celeste project, for example was ported from running on Cori to running on Microsoft Azure in less than one developer-week. That would be unthinkable with your traditional HPC application that ends up being deeply coupled to the precise system it was built to run on. A large part of why that was possible was the philosophy that Julia has of trying to provide the same programming experience everywhere. We're not unwilling to support this kind of use case but doing so is a fair amount of extra work and there is only so much time, money and manpower available so we have to pick and choose our battles. The battle we have chosen to fight first is to make Julia "just work" for the vast majority of users who do not run their code on supercomputers or need to link against specially tuned system libraries. This approach provides the most benefit to the most people. And indeed, Julia users love its package manager because they install packages and things mostly do just work. However funding is not a fixed thing. If the national labs have special requirements, perhaps there is funding available to help get the work done to support their special use cases. There are companies (Julia Computing is one such) who can help with this kind of work and are more than willing to upstream any improvements made to Julia's build systems to the open source project. |
|
We actively support julia users on several top HPC systems and I personally have talked extensively with the relevant sysadmins about requirements. As a result, I do think we understand the challenges involved in getting Julia running on systems of these sorts and indeed supporting this was an explicit design goal of the Pkg and BinaryBuilder work. However, that feature is not complete yet and will certainly not be started until after we have resolved JuliaLang/Pkg.jl#841. Our focus right now is to make julia work out of the box on as many systems as possible in order to target the most number of users possible, including users on HPC systems. We have made enormous progress in this regard to the point that julia does tend to "just work" in most places. Yes, that means that it is currently harder to support centrally administered software repositories (whether on HPC systems or in Linux distros, etc), but that is simply an indication that we haven't gotten to that part yet, because it's fairly niche for our user community (especially as julia does tend to work for these users, even if using slightly suboptimal library installations which may or may not matter). We will be able to resolve them in due time, but in the mean time, additional suggestions that we don't know what we're doing or don't care about our users are not particularly productive. |
|
@StefanKarpinski @Keno Thank you for your constructive responses. I understand your point now. Would you please leave this issue open, so that potential GNU/Linux distribution developers and users could find this bug easily? Denying the existance of this bug puts contributions off. In the free software community, people volunteer: funding is a good-to-have but not a must to work. I have seen heroic efforts from @cdluminate to introduce Julia to the Debian community, with millions of potential users. I could vision we work toward upstreaming @cdluminate's patches to honestly solve this bug. We care about Julia and we are willing to help. @Keno, just to comment that it is not "slightly suboptimal", the suboptimal library could easily drag performance down by 10x, e.g. in @casparvl's case. |
|
I have never seen a 10x difference in performance because of the way a library was configured or built—frankly, that’s a bit implausible. If that’s the case then either the default we ship should be much better or the “optimized” version is doing something very sketchy and wrong. |
Stefan, there is no need to limit our discussion within reconfiguring and rebuilding a library. This issue cares more about substituting a generic Julia-dependent library with a much more specialized, fine-tuned and optimized one. For example, for netlib blas-reference vs BLIS, the difference will be even more than 10x on exotic platforms like Intel Xeon Phi KNL. In general a 10x difference is not an exaggeration. Scientists and engineers in supercomputing centers work full time to tune performance for a reason. Apart from performance, portability and easiness for building from source is even more important to users. |
|
C'mon, man. Nobody is arguing that Julia users should use reference BLAS. Julia has never—even from the very earliest days—shipped with a reference BLAS because we're not incompetent and we care deeply about performance. A huge amount of work has been done to find and correctly configure high-quality, high-performance, portable libraries like OpenBLAS, dSFMT, etc. We even went so far as creating our own openlibm library because no existing libm at the time met the requirements of being fast, portable, open source and liberally licensed (since then many other projects have adopted openlibm). So talking about stuff like reference BLAS is just a straw man argument and a lecture about "scientists and engineers in supercomputing centers work full time to tune performance for a reason" is just obnoxious and unwarranted. Regarding Xeon Phi KNL, yes, that's what Cori has and @Keno got Julia running at 1.5 petaflops on that machine for the Celeste project. So we're well aware of what it takes to get Julia running on HPC systems. It's fine to discuss how to make it easier to use custom libraries on HPC systems but please stick to the actual situation rather than bogus claims that what we are shipping by default is 10x slower. |
|
@StefanKarpinski @Keno Thanks a lot for your constructive responses. First of all,
if that was how my post came accross, I apologize. I know that developers in general care a lot about their projects. Earlier in this thread, there was an argument that 'there are so many names for BLAS libraries', suggesting that this would be the problem - and that, I could not understand. But clearly, that is not the main issue. The main issue as I understand it now seems to be
and although I don't personally grasp the details of this statement (don't these libs have standard APIs?) I trust you know far better than I do.
This, clearly, is a fair design choice and far be it from me to argue with that. Apart from performance, I also encounter users with portability issues, so I understand the value of this design choice. I know that a lot of these things are a matter of priority, because resources are always limited. And, one key factor in determining priorities is to know what is important to your users (the exact same applies to us, as an HPC center by the way). I would like my above post to be a signal that yes, linking to low level libraries could be important to a (group of) users, namely those on university clusters & HPC systems. They may not be your biggest or most important users group - you probably know that better than I do - but running on such systems has some weird implications that do not apply to 'normal' users. For example, I do not assume that linking to system libraries will result in a 10x speedup in a typical case, but what you have to realize is that on a typical HPC system, if one can achieve a 5% speedup of all the software running on that system, that means one could save anywhere between tens of thousands and millions of USD/Euro in hardware, depending on the size of the system. Thus, even though a lot of software download instructions say 'just download the binaries, because it is easier', the HPC community tends to be willing to spend the extra effort and compile from source. In 99/100 cases, this is worthwhile because software developers themselves regularly focus on functionality, rather than performance. I'm glad to see that Julia is in the 1%. |
|
I have a question related to this thread. On one Mac, I have been building Arpack.jl successfully by installing By inspecting -- The Fortran compiler identification is GNU 7.3.0` in the first Mac as desired, but -- The Fortran compiler identification is GNU 8.1.0`in the second Mac. Apparently pointing Am I missing something? Any ideas? |
|
|
@staticfloat, on both Macs the directory of the homebrew-installed I even tried to link -- The Fortran compiler identification is GNU 8.1.0and I did Any ideas how to force the Fortran version to use for building |
|
Any ideas? |
|
@wsshin So that we don't bother other people, let's open a new issue. I will briefly note that I do not think you are actually compiling anything, since the log file location leads me to believe that you are instead getting some precompiled binaries. I'm willing to bet that you compiled |
I'm getting the same error 'julia: symbol lookup error: /home/puck/.julia/packages/Arpack/cu5By/deps/usr/lib/libarpack.so: undefined symbol: dlamch_64_' Did you ever fix it? |
|
@puckvg What's your configuration? |
|
This seems to also happen on Cirrus CI: |
|
I've found a workaround for those like @carstenbauer (and myself) who are on university clusters and can't simply make a symlink
The process could be upstreamed into Arpack.jl and make more reliable by automatically making the libarpack.so binary reference whatever |
and others
I tried installing Arpack on 0.7 beta and saw this error trying to run
eigs:To see if this was solved on master, I tried installing it. But this happens when I try to
develop Arpack#master:The text was updated successfully, but these errors were encountered: