libcxxwrap_julia: switch to libjulia

[fingolfin]

Also drop the restriction to the cxx11 ABI and add more platforms.

Motivation: while this is not the perfect next step for libcxxwrap_julia, I think it makes sense to do this now: switching to libjulia and adding more platforms will allow other packages using libcxxwrap_julia to do the same. Once (if?) we get "full" support for "julia_version" as part of the platform, adjusting libcxxwrap_julia and other JLLs depending on it to benefit from those shouldn't be hard; but it is unclear when this will be available; so why not implement those benefits we can already get now?

As a side effect, this will produce a fresh JLLWrapper based JLL :-)

However, this PR shouldn't be merged without @barche approving.

UPDATE: ... and of course it should pass the CI tests before all else ;-)

Close #1647

[benlorenz]

To fix mingw please try adding -DJulia_LIBRARY=$libdir/libjulia.$dlext (it shouldn't hurt setting this for other targets as well).
(FindJulia expects a libjulia.dll.a in $prefix/lib but libjulia only has the shared library libjulia.dll in $libdir == $prefix/bin)

And there is no libjulia_jll for freebsd (yet?):

Yggdrasil/L/libjulia/common.jl

Line 180 in fddff44

filter!(!Sys.isfreebsd, platforms)

[fingolfin]

This seems to work now! Thanks to @benlorenz for the help.

@barche care to take a look?

[barche]

This looks great to me, thanks for doing this. Just two doubts:

• I don't like the need to set -DJulia_LIBRARY manually, this will need to be done on all dependent packages. Should we fix this in FindJulia directly?
• I'd prefer to release a version based on libjulia1.3 and then the 1.4 one

[fingolfin]

Let's wait for PR #1951 and its two follow-ups, then I'll redo this PR .

[fingolfin]

Updated for that latest libjulia_jll. Unfortunately there is a compile error on FreeBSD: /workspace/destdir/include/julia/ios.h:7:10: fatal error: 'uv.h' file not found. But I guess we could go without FreeBSD support for now...

Regarding Julia_LIBRARY -- yeah it would be good if that could be fixed centrally, instead of forcing all other JLLs using libcxxwrap to adapt.

[giordano]

Updated for that latest libjulia_jll. Unfortunately there is a compile error on FreeBSD: /workspace/destdir/include/julia/ios.h:7:10: fatal error: 'uv.h' file not found. But I guess we could go without FreeBSD support for now...

We could go without FreeBSD (as long as @ararslan doesn't see us), but the error is rather weird 🤔

[fingolfin]

uv.h should come from LibUV_jll, and I just verified that it is in there, including the FreeBSD variant.

But I guess the compiler might not pick up the include path where uv.h has been installed to? But then why does it work on other platforms?

[barche]

Regarding Julia_LIBRARY -- yeah it would be good if that could be fixed centrally, instead of forcing all other JLLs using libcxxwrap to adapt.

With commit 2ee08fa0877f7268d06fd23c1be1b7c80b5de48c on libcxxwrap-julia it should work without the Julia_LIBRARY. Best to also bump the version to v0.8.2, which I will tag if this works out OK.

[benlorenz]

uv.h should come from LibUV_jll, and I just verified that it is in there, including the FreeBSD variant.

But I guess the compiler might not pick up the include path where uv.h has been installed to? But then why does it work on other platforms?

Probably because on FreeBSD (in general and not just in BB) <sysroot>/usr/local/include is not in the default path when searching headers and $destdir (containing all the dependencies) is linked to <sysroot>/usr/local.
I guess it would make sense to add that directory as a search path in the compiler wrappers for freebsd?

Otherwise adding -I$includedir would also help but then needs to be replicated in all other related build scripts.

sandbox:${WORKSPACE}/srcdir/build # /opt/bin/clang++ --sysroot=/opt/x86_64-unknown-freebsd11.1/x86_64-unknown-freebsd11.1/sys-root/  -DJLCXX_EXPORTS -DJULIA_ENABLE_THREADING -Dcxxwrap_julia_EXPORTS -I/workspace/srcdir/libcxxwrap-julia/include -I/workspace/destdir/include/julia  -Wunused-parameter -Wextra -Wreorder -O3 -DNDEBUG -fPIC   -std=gnu++17 -o CMakeFiles/cxxwrap_julia.dir/src/jlcxx.cpp.o -c /workspace/srcdir/libcxxwrap-julia/src/jlcxx.cpp -v
clang version 9.0.1 (/home/staticfloat/.julia/dev/BinaryBuilder/deps/downloads/llvm-project.git c1a0a213378a458fbea1a5c77b315c7dce08fd05)
Target: x86_64-unknown-freebsd11.1
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
 "/opt/x86_64-linux-musl/bin/clang-9" -cc1 -triple x86_64-unknown-freebsd11.1 -aux-triple x86_64-unknown-freebsd11.1 -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name jlcxx.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -coverage-notes-file /workspace/srcdir/build/CMakeFiles/cxxwrap_julia.dir/src/jlcxx.cpp.gcno -resource-dir /opt/x86_64-linux-musl/lib/clang/9.0.1 -D JLCXX_EXPORTS -D JULIA_ENABLE_THREADING -D cxxwrap_julia_EXPORTS -I /workspace/srcdir/libcxxwrap-julia/include -I /workspace/destdir/include/julia -D NDEBUG -isysroot /opt/x86_64-unknown-freebsd11.1/x86_64-unknown-freebsd11.1/sys-root/ -internal-isystem /opt/x86_64-unknown-freebsd11.1/x86_64-unknown-freebsd11.1/sys-root//usr/include/c++/v1 -O3 -Wunused-parameter -Wextra -Wreorder -std=gnu++17 -fdeprecated-macro -fdebug-compilation-dir /workspace/srcdir/build -ferror-limit 19 -fmessage-length 97 -fobjc-runtime=gnustep -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -fopenmp-is-device -faddrsig -o CMakeFiles/cxxwrap_julia.dir/src/jlcxx.cpp.o -x c++ /workspace/srcdir/libcxxwrap-julia/src/jlcxx.cpp
clang -cc1 version 9.0.1 based upon LLVM 9.0.1 default target x86_64-linux-musl
#include "..." search starts here:
#include <...> search starts here:
 /workspace/srcdir/libcxxwrap-julia/include
 /workspace/destdir/include/julia
 /opt/x86_64-unknown-freebsd11.1/x86_64-unknown-freebsd11.1/sys-root//usr/include/c++/v1
 /opt/x86_64-linux-musl/lib/clang/9.0.1/include
 /opt/x86_64-unknown-freebsd11.1/x86_64-unknown-freebsd11.1/sys-root//usr/include
End of search list.
In file included from /workspace/srcdir/libcxxwrap-julia/src/jlcxx.cpp:1:
In file included from /workspace/srcdir/libcxxwrap-julia/include/jlcxx/array.hpp:4:
In file included from /workspace/srcdir/libcxxwrap-julia/include/jlcxx/type_conversion.hpp:4:
In file included from /workspace/destdir/include/julia/julia.h:12:
In file included from /workspace/destdir/include/julia/libsupport.h:13:
/workspace/destdir/include/julia/ios.h:7:10: fatal error: 'uv.h' file not found
#include "uv.h"
         ^~~~~~
1 error generated.

[fingolfin]

Huh, OK, interesting. So how come other packages which rely on headers from JLLs installed as (build) dependency do compile on FreeBSD? Are they all already adding in explicit -I flags?

[fingolfin]

With commit 2ee08fa0877f7268d06fd23c1be1b7c80b5de48c on libcxxwrap-julia it should work without the Julia_LIBRARY. Best to also bump the version to v0.8.2, which I will tag if this works out OK.

@barche I switched to that commit and it seems to work as intended

[benlorenz]

Huh, OK, interesting. So how come other packages which rely on headers from JLLs installed as (build) dependency do compile on FreeBSD? Are they all already adding in explicit -I flags?

In some cases cmake or configure might determine that these are needed, so this could probably be fixed in FindJulia as well. But it is also given in the documentation that it might be needed in some cases (a quick grep found about 100 such flags in the recipes): https://juliapackaging.github.io/BinaryBuilder.jl/dev/troubleshooting/#Header-files-of-the-dependencies-can't-be-found
For LDFLAGS there is even a comment that this was needed just for FreeBSD in one case.

In libjulia there is also CFLAGS="-I${prefix}/include" and LDFLAGS="-L${prefix}/lib".

[barche]

I hope it's not rude to push changes, but I think with the latest update of FindJulia it will work. Fingers crossed.

[fingolfin]

Not rude at all, to the contrary; in fact, I was somewhat worried when I made this that you'd find me rude for doing a PR for "your" JLL... but I figured it'd be useful to get this kicked off to find obstacles and remove them, and, well, that's exactly what happened ;-). In fact, I am somewhat worried whether the JLL this PR produces works in practice -- so I think before this PR is merged, we really should test it locally on some systems (and even before that, perhaps download the tarballs created by the CI runners and inspect the binaries in them).

[benlorenz]

I did a first test with the x86_64-linux-gnu.tar.gz tarball and my julia 1.5 installation: building the latest libpolymake-julia using this tree worked fine and also the Polymake.jl testsuite had no errors (using this tarball via Overrides.toml, i.e. using the old wrappers but the new binaries).

[fingolfin]

Since this seems to work well now, how shall we proceed? Merge this, then @barche proceeds to make Julia 1.4 and 1.5 variants?

[barche]

Since this seems to work well now, how shall we proceed? Merge this, then @barche proceeds to make Julia 1.4 and 1.5 variants?

Yes, that's the idea, but this also reminds me: wasn't there a way to limit the Julia compat in the JLL? It would be better to have this done in the JLL rather than in CxxWrap.jl. In that case, the limit for this JLL should be Julia 1.3, but I forgot how this is done.

[fingolfin]

Using julia_compat, you can grep Yggdrasil for usage examples.

But! there is a caveat: packages with the the same version, just different build version (the +3 bit at the end of JLL versions) can't really have different compat restrictions. So you'll need to use different versions for this to work, even if they only differ in the patch level... might be a good idea to check back with @giordano and @staticfloat , too

[fingolfin]

Perhaps one could abuse the prerelease part of the version string? I.e. use versions 0.8.3-julia.1.3, 0.8.3-julia.1.4, ... ?

[fingolfin]

I've rebased and squashed this.

I've also added a julia_compat field and added the version number tweak (abusing the "prerelease" feature of semver). So in principle, we should now be able to build separate versions for Julia 1.3, 1.4, 1.5 and 1.6 (though that'll require #1987 to be merged).
Alternatively, one could hope that e.g. the 1.5 build will be compatible with 1.6 and later builds, and drop its compat restriction / change it to ">= 1.5". But that seems risky for a minuscule gain (one less JLL version to be build), yet requires extra work (tweaking the julia compat just for the 1.5 build), which in turn increases the risk of making a mistake when building this.

Another thing: We could split this recipe now into multiple (like we did for e.g. libjulia), that coexist; that might make maintenance a tad easier in the future? (Or not, dunno)

[fingolfin]

OK, all working well now :-).

[barche]

This looks good to me!

[giordano]

What's the rationale of this version number? Pkg doesn't really like anything outside of X.Y.Z and we already overstress it with the build number.

[fingolfin]

Here's the reason: We need to build separate binaries (artifacts) of libcxxwrap for *at least Julia 1.3 versus Julia >= 1.4; but better, for each of Julia 1.3, 1.4, 1.5, 1.6. Each of the corresponding JLL wrappers needs a different compat entry for Julia in Project.toml. But to do that, they need actual different versions (the build number part after the + is not enough for that, the registry doesn't support that, which makes sense, as it is consistent with how semver defines build number).

Unlike build numbers, the pre-release is part of semver which is explicitly defined to indicate different versions, including how they should be compared with each other and other versions. It also is used by Julia itself As such, I would hope that existing Julia tooling correctly deals with it (if not, that'd be a bug in the semver support of that tool, which IMHO should be fixed).

One drawback is that the semver spec says this "A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version." However, that should not matter for any tooling, it's just something about the semantics for users; and the only user of this JLL would be CxxWrap.jl, so there should be no issue here, I think.

[barche]

I think this is OK, as long as it works. Not knowing about pre-releases, what I had in mind for this was:

• libcxxwrap 0.8.2 = Julia 1.3
• libcxxwrap 0.8.3 = Julia 1.4
• libcxxwrap 0.8.4 = Julia 1.5
• libcxxwrap 0.8.5 = Julia 1.6

Starting from Julia 1.6, I am hoping this is solved by the julia version being part of the artifact resolution, if I understood correctly?

[giordano]

Starting from Julia 1.6, I am hoping this is solved by the julia version being part of the artifact resolution, if I understood correctly?

Well, that would work for Julia v1.6+

[fingolfin]

@barche if you are willing to go through that hassle, sure, why not (but perhaps shift all the libcxxwrap version up by one, then the last digits of the libcxxwrap and Julia version match, that might make things easier?).

Of course it also raises the question how you then would version a bugfix release? Perhaps by adding 10 to the patchlevel each time?

• libcxxwrap 0.8.3, 0.8.13, 0.8.23, ... = Julia 1.3
• libcxxwrap 0.8.4, 0.8.14, 0.8.24, ... = Julia 1.4
• libcxxwrap 0.8.5, 0.8.15, 0.8.25, ... = Julia 1.5
• libcxxwrap 0.8.6, 0.8.16, 0.8.26, ... = Julia 1.6

[fingolfin]

? A single version wouldn't allow to produce several binaries for several Julia versions, though? And hence wouldn't allow to resolve the crashes with Julia 1.3.

Also, the current release is 0.8.0, hence this should be at least 0.8.1.

It is also far from clear to me that this is a temporary situation which will go away with Julia 1.6 (plus, it will likely be months until then, won't it? : first off, that assumes everyone is ready to immediately drop support for all Julia versions once 1.6.0 is released. Secondly, it is not yet clear to me that Julia 1.6 will really resolve this: are all pieces already there? Even if, without verifying that it works as desired in practice, I'd be reluctant to rely on that (no offense intended to @staticfloat and anybody else who may habe worked on this).

[barche]

No, what I mean is keep the proposed mapping as:

• libcxxwrap 0.8.2 = Julia 1.3
• libcxxwrap 0.8.3 = Julia 1.4
• libcxxwrap 0.8.4 = Julia 1.5
• libcxxwrap 0.8.5 = Julia 1.6

If you think bugfixes are important for older Julia versions, we could do:

• libcxxwrap 0.8.2 = Julia 1.3
• libcxxwrap 0.9.0 = Julia 1.4
• libcxxwrap 0.10.0 = Julia 1.5
• libcxxwrap 0.11.0 = Julia 1.6

[fingolfin]

As long as they are all using the same source, I'd suggest to keep them all at 0.8.x

[fingolfin]

I've changed the version (back) to 0.8.2, as discussed here. Can this be merged now, or is anything still to be done? (I am not aware of anything, but...)

[barche]

OK for me!