Latest version doesn't load with pre-compiled version of julia

[jstrube]

I'm on 1b1980e
Building with cmake and gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008 is fine, but then loading with a downloaded version of julia-1.3.1 fails with

julia> dlopen("libcxxwrap_julia")
ERROR: could not load library "libcxxwrap_julia"
/home/jstrube/Applications/julia-1.3.1/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by libcxxwrap_julia.so)
Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(dlopen), ::String, ::UInt32) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109
 [2] dlopen at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)
 [3] top-level scope at REPL[5]:1

This works fine with a version of julia that I've built myself...

[barche]

Yes, I get the same on nightly. I know of only three possible solutions:

• use the julia supplied by the linux distribution
• compile Julia from source
• compile inside BinaryBuilder

I think this is because the C++ stdlib that is shipped with Julia is of a different version now, but I don't know why or when exactly this changed.

CC @staticfloat @giordano in case they have another workaround

[jstrube]

The first solution would be the worst, the third solution the best, IMHO. I understand the difficulties doing this in BB, though.

…

On Thu, Jan 2, 2020, 12:03 AM Bart Janssens ***@***.***> wrote: Yes, I get the same on nightly. I know of only three possible solutions: - use the julia supplied by the linux distribution - compile Julia from source - compile inside BinaryBuilder I think this is because the C++ stdlib that is shipped with Julia is of a different version now, but I don't know why or when exactly this changed. CC @staticfloat <https://github.com/staticfloat> @giordano <https://github.com/giordano> in case they have another workaround — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#39?email_source=notifications&email_token=AACX57UXCZDXNHABEBLJJEDQ3WNU5A5CNFSM4KBZ7WO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH52I4Y#issuecomment-570139763>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACX57WWTR2LJEYK6WTU6WLQ3WNU5ANCNFSM4KBZ7WOQ> .

[barche]

To be clear, I was talking mostly about easing local development of CxxWrap-based packages. For final distribution, BinaryBuilder is a must, since that's the way to guarantee compatibility with the official binaries.

For local development, what could possibly also work is to switch to the clang compiler for CxxWrap and its dependents, I havent't tested this but it should avoid the conflict with the GNU stdlib.

[jstrube]

For local development, I'm ok with building Julia from source. It's pretty quick these days.

[j-fu]

Hi, just came over here by chance...

I have a similar problem on OpenSUSE tumbleweed (which BTW has a decent system package, but still at 1.2). What works for me is to change the CXX lib used in Julia to the one from the system (which is assumed by your compiled code). Find the stdc++ library used by g++ in your system
(for me it is /usr/lib64/libstdc++.so.6) and start Julia as follows:

$ LD_PRELOAD=/usr/lib64/libstdc++.so.6 julia

Jürgen

[benlorenz]

We got hit by this as well and there is a roadmap for adressing this here: JuliaLang/julia#34276

The easiest workaround is probably to just delete libstdc++ from the julia lib directory. The more recent one from the system will be used instead and is backwards compatible.

[jstrube]

Is this error related?
C++ exception while wrapping module LCIO: StlWrapper was not instantiated
I just came across it now when trying to use binaries from Yggdrasil. If this is separate, I'll open a new issue.

[findmyway]

@jstrube

Did you run @wrapmodule in the Main REPL?

I have the same error when trying to use binaries from Yggdrasil in the Main REPL. But if I wrap it in a module, the error is gone.

[jstrube]

Yeah, I'm not running this interactively...
It's strange, it works on one machine (Ubuntu 19.10), but not the other (RHEL6). My code with the failed build is here JuliaRegistries/General#9325
I'm using the exact same pattern (at least I thought I was) in a different package, which was just successfully merged. JuliaRegistries/General#9331
I can maybe understand why it works on Ubuntu 19.10, but not on RHEL6. RHEL6 is just ancient...
What I don't understand is why it works on one build server, but not on another one. On my RHEL6 machine, both packages throw the same error...

Can you explain what exactly you are wrapping in the REPL? Am I missing something in the LCIO package?

[findmyway]

Oh, I just find that it has nothing to do with whether to wrap it in a module or not...

What I have done previously:

(v1.3) pkg> add https://github.com/findmyway/OpenSpiel_jll.jl

julia> using OpenSpiel_jll
[ Info: Precompiling OpenSpiel_jll [bd10a763-4654-5023-a028-c4918c6cd33e]

julia> using CxxWrap
[ Info: Precompiling CxxWrap [1f15a43c-97ca-5a2a-ae31-89f07a497df4]

julia> @wrapmodule(libspieljl)
C++ exception while wrapping module Main: StlWrapper was not instantiated
ERROR: StlWrapper was not instantiated
Stacktrace:
 [1] register_julia_module at /home/tj/.julia/packages/CxxWrap/lDNAy/src/CxxWrap.jl:360 [inlined]
 [2] readmodule(::String, ::Symbol, ::Module, ::Nothing) at /home/tj/.julia/packages/CxxWrap/lDNAy/src/CxxWrap.jl:664
 [3] wrapmodule(::String, ::Symbol, ::Module, ::Nothing) at /home/tj/.julia/packages/CxxWrap/lDNAy/src/CxxWrap.jl:670
 [4] top-level scope at REPL[7]:1

BUT, if I execute using CxxWrap first, the error is gone.

julia> using CxxWrap

julia> using OpenSpiel_jll

julia> @wrapmodule(libspieljl)

[jstrube]

Hmmm. Interesting...
My depsfile has using CxxWrap before anything else, but I'll play with some iterations of this tomorrow. Thanks for the tip.

[jstrube]

I've tried a couple different combinations in terms of load order, but I haven't found one that works.
I should also note that for me, the error occurs in @initcxx.
But... This is for testing.
In my running code, if I do using CxxWrap before using LCIO, now it magically works, even though the depsfile contains using CxxWrap... Very confusing.

[barche]

Strange, this error happens when __init__ for the StdLib submodule is never called, but this should be called automatically if you do using CxxWrap. To debug this further it could be useful to add a println to StdLib.__init__to see if it really is called.

[jstrube]

Good idea. I tried. It does get called, but the error still occurs. This makes sense, because it does work on a different system.
This does seem to be related to the distributed binaries somehow, but I still don't really understand it. Will keep trying to add more print statements.

(v1.3) pkg> test LCIO
   Testing LCIO
 Resolving package versions...
    Status `/tmp/jl_1TSBt1/Manifest.toml`
  [b99e7846] BinaryProvider v0.5.8
  [1f15a43c] CxxWrap v0.9.0 [`~/.julia/dev/CxxWrap`]
  [864edb3b] DataStructures v0.17.9
  [94167541] LCIO v1.3.0
  [1fe59b3e] LCIO_Julia_Wrapper_jll v0.7.0+0
  [35fd387a] LCIO_jll v2.13.1+0
  [1914dd2f] MacroTools v0.5.4
  [bac558e1] OrderedCollections v1.1.0
  [83775a58] Zlib_jll v1.2.11+8
  [3eaa8342] libcxxwrap_julia_jll v0.6.6+0
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]
Hi, you just called StdLib.__init__()
Hi, you just called StdLib.__init__()
C++ exception while wrapping module LCIO: StlWrapper was not instantiated
ERROR: LoadError: InitError: StlWrapper was not instantiated
Stacktrace:
 [1] register_julia_module at /afs/desy.de/user/j/jstrube/.julia/dev/CxxWrap/src/CxxWrap.jl:360 [inlined]
 [2] register_julia_module(::Module) at /afs/desy.de/user/j/jstrube/.julia/dev/CxxWrap/src/CxxWrap.jl:367
 [3] __init__() at /afs/desy.de/user/j/jstrube/.julia/packages/LCIO/glZVz/src/LCIO.jl:11
 [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:692
 [5] _require_from_serialized(::String) at ./loading.jl:743
 [6] _require(::Base.PkgId) at ./loading.jl:1034
 [7] require(::Base.PkgId) at ./loading.jl:922
 [8] require(::Module, ::Symbol) at ./loading.jl:917
 [9] include at ./boot.jl:328 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1105
 [11] include(::Module, ::String) at ./Base.jl:31
 [12] include(::String) at ./client.jl:424
 [13] top-level scope at none:6
during initialization of module LCIO
in expression starting at /afs/desy.de/user/j/jstrube/.julia/packages/LCIO/glZVz/test/runtests.jl:1
ERROR: Package LCIO errored during testing

[barche]

I think this might actually be related to the flags used for dlopen ( see JuliaLang/julia#2312 ), is this only happening on Linux and working fine on Mac?

You could add the flags RTLD_NOLOAD | RTLD_GLOBAL to the wrapmodule call in LCIO.jl, I think it would look like this:

@wrapmodule(lciowrap, :define_julia_module, Libdl.RTLD_NOLOAD | Libdl.RTLD_GLOBAL)

Also, for the same reason, whatever you do, don't import or use libcxxwrap_julia_jll, it will dlopen the stl library automatically with the wrong flags. This is fixed in the upcoming version 0.7 of libcxxwrap_julia_jll.

[jstrube]

That's an interesting suggestion, but it doesn't help in this case, unfortunately. ] test CxxWrap works fine, but test LCIO still fails with the same error, even after changing the @wrapmodule call.
Building LCIO against downloaded versions of libcxxwrap_julia has been working, so I could do that again, but it's only the test that fails. using CxxWrap followed by using LCIO is just fine, and the code seems to run without issues. I've also tried using Pkg; pkg"test LCIO" on the REPL after using CxxWrap, but that also doesn't work.
Not sure about Mac. For my FastJet package, actually Linux works and Mac doesn't... https://travis-ci.com/jstrube/FastJet.jl/jobs/287657149
For LCIO, neither Mac not Linux work: https://travis-ci.com/jstrube/LCIO.jl

[jstrube]

Is this related? JuliaPackaging/BinaryBuilder.jl#453

[barche]

Yes, it is. I was just experimenting a bit, since I could reproduce this easily on my machine, and manually changing the dlopen line in the LCIO_Julia_Wrapper_jll package to this fixed the problem:

global lciowrap_handle = dlopen(lciowrap_path, Libdl.RTLD_GLOBAL)

Unfortunately I couldn't find any other short-term workaround.

[jstrube]

Interesting... Thanks for finding that! Looks like I need to make a new release again soon.

…

On Sun, Feb 16, 2020, 2:41 PM Bart Janssens ***@***.***> wrote: Yes, it is. I was just experimenting a bit, since I could reproduce this easily on my machine, and manually changing the dlopen line in the LCIO_Julia_Wrapper_jll package to this fixed the problem: global lciowrap_handle = dlopen(lciowrap_path, Libdl.RTLD_GLOBAL) Unfortunately I couldn't find any other short-term workaround. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39?email_source=notifications&email_token=AACX57UDIYWMU7UNET7BMX3RDG6KJA5CNFSM4KBZ7WO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL4UXFI#issuecomment-586763157>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACX57WFYV4QCAMKK5Z6D2TRDG6KJANCNFSM4KBZ7WOQ> .

[barche]

OK, manually adding the line

Libdl.dlopen("/home/user/tmp/cleandepot/artifacts/0d6202473ddcca22a6d7c6c527d2ccd99884412f/lib/libcxxwrap_julia_stl.so", Libdl.RTLD_GLOBAL | Libdl.RTLD_NOLOAD, throw_error = false)

to __init__ in the CxxWrapCore module works around the problem. I think I could generalize this, but only for Julia 1.3 and higher.

The problem is that the libraries get loaded from the precompiled code, which disregards the fact that you first import CxxWrap, which in the non-precompiled case should be enough to avoid this too.

[jstrube]

Cool, thank you! I'd be ok with requiring Julia 1.3+ from the next version.

[barche]

I just tried with CxxWrap master and the experimental libcxxwrap 0.7 binaries at https://github.com/barche/libcxxwrap_julia_jll.jl and it appears to work without dlopen tricks.

[ordicker]

sorry to bump this issue, but I followed the README with julia-1.5.1 (ubuntu 20.4) and got the same error.
This patch solve it:
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 julia

Am I doing something wrong or this bug isn't solved yet on the master?

[barche]

sorry to bump this issue, but I followed the README with julia-1.5.1

Do you mean that this happens when you compile libcxxwrap-julia yourself? If so, it is expected, the easiest workaround is to remove libstdc++ from the Julia lib directory, or also compile Julia itself from source.

It's also possible to adopt a container-based workflow now, this is demonstrated in the JuliaCon2020 workshop: https://live.juliacon.org/talk/NNVQQF

[ordicker]

I did compile it me self (not sure why but I'm following this )

Thanks for your response

[barche]

Ah, yes, for now OpenCV requires building everything from source, until someone builds OpenCV binaries using BinaryBuilder.