Malloc problems

[moble]

Description

This problem isn't specific to this package (or even Julia), but I want to be able to track it in one place. In our collaboration, certain clusters have LD_PRELOAD set to load jemalloc, which evidently also loads some libc that is ABI incompatible with Julia, leading to a double-free and crash when attempting to install Julia. This also happens with python — for example, when importing numba — which is evidently why a custom python build is required for integration with the spectre code. Unfortunately, this issue suggests that even a source build won't help...

Meanwhile, here are some references:

• Julia crashes with "free(): invalid pointer" or "double free or corruption (out)" on ubuntu 20.04.1 TPU vm JuliaLang/julia#44242
• Preloading problem with jemalloc on glibc ≥ 2.34 JuliaLang/julia#46298
• Broken inside Google Colab JuliaLang/julia#48461
• Provide an error message if LD_PRELOAD is set during a Julia build or startup JuliaLang/julia#55487
• Colab update breaks compatibility with JuliaLang googlecolab/colabtools#3385 (comment)
• https://discourse.julialang.org/t/issues-with-julia-installation-on-google-tpu-vm/65783/10

And here's the original error:

build-Release]$ ./bin/python-spectre -c "from sxs import julia"
[juliapkg] Found dependencies: sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/sxs/juliapkg.json
[juliapkg] Found dependencies: sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliacall/juliapkg.json
[juliapkg] Found dependencies: sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliapkg/juliapkg.json
[juliapkg] Locating Julia =1.10.0, ^1.10.3
[juliapkg] Querying Julia versions from https://julialang-s3.julialang.org/bin/versions.json
[juliapkg] Using Julia 1.10.4 at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/julia
[juliapkg] Using Julia project at sxs/envs/spectre-python-2024-04/julia_env
[juliapkg] Installing packages:
           julia> import Pkg
           julia> Pkg.Registry.update()
           julia> Pkg.add([Pkg.PackageSpec(name="PostNewtonian", uuid="377afc40-5642-4616-8613-b7ebca523866"), Pkg.PackageSpec(name="PythonCall", uuid="6099a3de-0909-46bc-b1f4-468b9a2dfc0d")])
           julia> Pkg.resolve()
           julia> Pkg.precompile()
┌ Warning: attempting to remove probably stale pidfile
│   path = "/home/user/.julia/registries/.pid"
└ @ FileWatching.Pidfile sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:273
double free or corruption (out)

[237872] signal (6.-6): Aborted
in expression starting at none:1
__pthread_kill_implementation at /lib64/libc.so.6 (unknown line)
raise at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
__libc_message.cold at /lib64/libc.so.6 (unknown line)
malloc_printerr at /lib64/libc.so.6 (unknown line)
_int_free at /lib64/libc.so.6 (unknown line)
free at /lib64/libc.so.6 (unknown line)
Curl_parsenetrc at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/../lib/julia/libcurl.so.4 (unknown line)
Curl_connect at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/../lib/julia/libcurl.so.4 (unknown line)
multi_runsingle at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/../lib/julia/libcurl.so.4 (unknown line)
multi_socket at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/../lib/julia/libcurl.so.4 (unknown line)
curl_multi_socket_action at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/../lib/julia/libcurl.so.4 (unknown line)
curl_multi_socket_action at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/Curl.jl:51 [inlined]
curl_multi_socket_action at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/Curl.jl:59 [inlined]
macro expansion at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/utils.jl:56 [inlined]
do_multi at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/Multi.jl:129
#48 at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/Multi.jl:147 [inlined]
lock at ./lock.jl:229
#47 at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/Downloads/src/Curl/Multi.jl:144 [inlined]
#726 at ./asyncevent.jl:306
jfptr_YY.726_75831.1 at sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
start_task at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/task.c:1238
Allocations: 613063 (Pool: 612289; Big: 774); GC: 1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/sxs/julia/__init__.py", line 5, in <module>
    import juliacall
  File "sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliacall/__init__.py", line 287, in <module>
    init()
  File "sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliacall/__init__.py", line 159, in init
    CONFIG['exepath'] = exepath = juliapkg.executable()
                                  ^^^^^^^^^^^^^^^^^^^^^
  File "sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliapkg/deps.py", line 364, in executable
    resolve()
  File "sxs/envs/spectre-python-2024-04/lib/python3.11/site-packages/juliapkg/deps.py", line 336, in resolve
    run(
  File "/software9/spack/opt/spack/linux-rhel9-x86_64/gcc-13.2.0/python-3.11.6-fh6i4o3zgqejtt333ylcva2563o6vskd/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sxs/envs/spectre-python-2024-04/julia_env/pyjuliapkg/install/bin/julia', '--project=sxs/envs/spectre-python-2024-04/julia_env', '--startup-file=no', '-e', 'import Pkg; Pkg.Registry.update(); Pkg.add([Pkg.PackageSpec(name="PostNewtonian", uuid="377afc40-5642-4616-8613-b7ebca523866"), Pkg.PackageSpec(name="PythonCall", uuid="6099a3de-0909-46bc-b1f4-468b9a2dfc0d")]); Pkg.resolve(); Pkg.precompile()']' died with <Signals.SIGABRT: 6>.