Julia v0.7 is "linked" to unversioned libraries

[ronisbr]

Hi guys!

I am creating the package for julia 0.7 / 1.0 in openSUSE. However, I notice that it requires some unversioned library files inside the directory /usr/lib64/julia. I am not really sure if this is a good thing. rpmlint is giving me the following errors:

[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libamd.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libcamd.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libccolamd.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libcholmod.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libcolamd.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libdSFMT.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libgit2.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libgmp.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libmpfr.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libopenblas.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libpcre2-8.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libspqr.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libsuitesparseconfig.so
[ 1990s] julia.x86_64: E: devel-file-in-non-devel-package (Badness: 50) /usr/lib64/julia/libumfpack.so

All those files are symbolic links to the one in /usr/lib64. Question: can we change this so that the library version is included in the name?

[cdluminate]

@ronisbr FYI http://debomatic-amd64.debian.net/distribution#unstable/julia/0.7.0-1/contents

Debian's julia package ships a set of symlinks (no soversion in name) which point to versioned libraries. Go to the bottom part of the above link and you will find it out. I haven't observed problem so far.

[ronisbr]

In openSUSE, libraries links without soversion are consider development files. In fact, in Julia v0.7, those symlinks are using the versioned files (like libamd.so.2).

Moreover, I am not sure if it is good to link Julia to unversioned libraries. Let's say a library receives a major update that broke API. Then, Julia will also break until it is built against this new version.

Hence, the devs already took care of this (since the aforementioned symlinks are pointing to versioned libraries). However, I am wondering if those symlinks can be renamed to contain the version.

[cdluminate]

@ronisbr First, Julia will try to load (ccall) a shared object from the private shared object directory /usr/lib/*/julia . It won't break as long as the symlinks points to the correct versioned shared object.

Second, libjulia.so is not linked against the bunch of shared objects. Julia use them via dlopen.

~/b/j/lib ❯❯❯ readelf -d libjulia.so.1.0

Dynamic section at offset 0x239510 contains 35 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libLLVM-6.0.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000000e (SONAME)             Library soname: [libjulia.so.1]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/julia:$ORIGIN]

Third, I don't think unversioned shared objects or symlinks in any private directory should be considered as any kind of development file. Debian's package checker lintian won't complain about it. So I think the devel-file-in-non-devel-package error can be ignored or overriden.

[ronisbr]

Hi @cdluminate ,

Third, I don't think unversioned shared objects or symlinks in any private directory should be considered as any kind of development file. Debian's package checker lintian won't complain about it. So I think the devel-file-in-non-devel-package error can be ignored or overriden.

OK, I will verify what are the reasons for this warning with openSUSE devs. Thanks!

[cdluminate]

@ronisbr In case if you need a citation, have a look at this part of Debian policy https://www.debian.org/doc/debian-policy/upgrading-checklist.html#version-4-1-2 and the 10.2 chapter.

And this is the content of Debian's libjulia0.7 package:

~ ❯❯❯ dpkg -L libjulia0.7
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/julia
/usr/lib/x86_64-linux-gnu/julia/libccalltest.so
/usr/lib/x86_64-linux-gnu/julia/libsuitesparse_wrapper.so
/usr/lib/x86_64-linux-gnu/julia/sys.so
/usr/lib/x86_64-linux-gnu/libjulia.so.0.7.0
/usr/share
/usr/share/doc
/usr/share/doc/libjulia0.7
/usr/share/doc/libjulia0.7/NEWS.Debian.gz
/usr/share/doc/libjulia0.7/changelog.Debian.gz
/usr/share/doc/libjulia0.7/changelog.gz
/usr/share/doc/libjulia0.7/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libjulia0.7
/usr/lib/x86_64-linux-gnu/julia/.so
/usr/lib/x86_64-linux-gnu/julia/libLLVM.so
/usr/lib/x86_64-linux-gnu/julia/libamd.so
/usr/lib/x86_64-linux-gnu/julia/libcamd.so
/usr/lib/x86_64-linux-gnu/julia/libccolamd.so
/usr/lib/x86_64-linux-gnu/julia/libcholmod.so
/usr/lib/x86_64-linux-gnu/julia/libcolamd.so
/usr/lib/x86_64-linux-gnu/julia/libcurl.so
/usr/lib/x86_64-linux-gnu/julia/libdSFMT.so
/usr/lib/x86_64-linux-gnu/julia/libgit2.so
/usr/lib/x86_64-linux-gnu/julia/libgmp.so
/usr/lib/x86_64-linux-gnu/julia/libmbedcrypto.so
/usr/lib/x86_64-linux-gnu/julia/libmbedtls.so
/usr/lib/x86_64-linux-gnu/julia/libmbedx509.so
/usr/lib/x86_64-linux-gnu/julia/libmpfr.so
/usr/lib/x86_64-linux-gnu/julia/libopenblas.so
/usr/lib/x86_64-linux-gnu/julia/libopenlibm.so
/usr/lib/x86_64-linux-gnu/julia/libopenspecfun.so
/usr/lib/x86_64-linux-gnu/julia/libpcre2-8.so
/usr/lib/x86_64-linux-gnu/julia/libspqr.so
/usr/lib/x86_64-linux-gnu/julia/libssh2.so
/usr/lib/x86_64-linux-gnu/julia/libsuitesparseconfig.so
/usr/lib/x86_64-linux-gnu/julia/libumfpack.so
/usr/lib/x86_64-linux-gnu/julia/libunwind.so
/usr/lib/x86_64-linux-gnu/julia/libutf8proc.so
/usr/lib/x86_64-linux-gnu/libjulia.so.0.7

[cdluminate]

/usr/lib/x86_64-linux-gnu/julia/.so

Oops ... I caught a package bug ... Orz

[vtjnash]

Moreover, I am not sure if it is good to link Julia to unversioned libraries. Let's say a library receives a major update that broke API. Then, Julia will also break until it is built against this new version.

Yes, exactly. I added these indirection files specifically to avoid this and be friendlier to distribution package developers such as yourself (cf. fix #24796 and issue #6742 and announcement at https://discourse.julialang.org/t/psa-for-julia-system-package-builders/7827). Since we're calling dlopen on the files, these links give us a level of indirection to ensure that we pick the specific version during compilation, and then will always see that file first in our $ORIGIN path. (as a side effect, it also allows replacement of these files to redirect the dlopen calls, in the event that the distribution wants to update it to a new compatible version).

[ronisbr]

Thanks @vtjnash

I will report your rationale to openSUSE devs so that those "errors" can be neglected.