libunwind build system not picking up LDFLAGS causing build failure on systems without zlib

[NickCao]

LDFLAGS is required for the linker to find bundled libraries

julia/Make.inc

Line 728 in 378f192

LDFLAGS += -L$(build_libdir) -Wl,-rpath,$(build_libdir)

But libunwind build system is not passing it to the linker, causing configure/build failure on systems without zlib, for example the nixpkgs stdenv.

configure:3928: checking whether the C compiler works
configure:3950: gcc -m64   -U_FORTIFY_SOURCE -fPIC -lz     -Wl,-rpath,'$$ORIGIN' -Wl,-z,origin -Wl,-rpath-link,/build/julia-1.10.5/usr/lib -Wl,--enable-new-dtags  conftest.c  >&5
/nix/store/x7yyxvwy1f9hlx72rzrgx069jyf7hxwr-binutils-2.42/bin/ld: cannot find -lz: No such file or directory

Additionally, in #42782, -lz was added to LIBUNWIND_CFLAGS in deps/unwind.mk:

julia/deps/unwind.mk

Line 6 in 378f192

LIBUNWIND_CFLAGS := -U_FORTIFY_SOURCE $(fPIC) -lz $(SANITIZE_OPTS)

But -lz is already handled by libunwind configure script:
https://github.com/libunwind/libunwind/blob/v1.5.0/configure.ac#L316

[giordano]

@NickCao would you be able to give #55624 a try?

[giordano]

LDFLAGS is required for the linker to find bundled libraries

julia/Make.inc

Line 728 in 378f192

LDFLAGS += -L$(build_libdir) -Wl,-rpath,$(build_libdir)

Why do you claim setting LDFLAGS is necessary? Autotools should be able to find libraries in the prefix, no? That's what we do with BinaryBuilder (which also uses a non-standard prefix where to install the packages) and we never have to set LDFLAGS separately to point the linker to the prefix. And we already set the prefix in

julia/deps/tools/common.mk

Line 7 in 378f192

CONFIGURE_COMMON = --prefix=$(abspath $(build_prefix)) --build=$(BUILD_MACHINE) --libdir=$(abspath $(build_libdir)) --bindir=$(abspath $(build_depsbindir)) $(CUSTOM_LD_LIBRARY_PATH)

and you can see that's set also for you in #55624 (comment)

[NickCao]

Why do you claim setting LDFLAGS is necessary? Autotools should be able to find libraries in the prefix, no?

I believe the --prefix option is only about the installation prefix of the package currently being configured (libunwind)? It's not passed to the underlying compiler/linker by autotools, so the linker would still follow its default search paths.

[giordano]

I'm not sure: in BinaryBuilder we don't have the prefix's libdir (/workspace/destdir/lib) anywhere in the default search path of the compiler (see LIBRARY_PATH):

sandbox:${WORKSPACE} # SUPER_VERBOSE=1 gcc -v
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -D_GLIBCXX_USE_CXX11_ABI=1 -frandom-seed=0x75262c83 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -Wl,-rpath-link,/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -march=x86-64 -mtune=generic -v
Using built-in specs.
COLLECT_GCC=/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/lto-wrapper
Target: x86_64-linux-gnu
Configured with: /workspace/srcdir/gcc-5.2.0/configure --prefix=/workspace/destdir --target=x86_64-linux-gnu --host=x86_64-linux-musl --build=x86_64-linux-musl --disable-multilib --disable-werror --enable-shared --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-gnu/sys-root --program-prefix=x86_64-linux-gnu- --disable-bootstrap --with-arch=x86-64 --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 5.2.0 (GCC) 
COMPILER_PATH=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/:/opt/x86_64-linux-gnu/bin/../libexec/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/bin/
LIBRARY_PATH=/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/:/opt/x86_64-linux-gnu/bin/../lib/gcc/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/:/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/:/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/
COLLECT_GCC_OPTIONS='-D' '_GLIBCXX_USE_CXX11_ABI=1' '-frandom-seed=0x75262c83' '-L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64' '-march=x86-64' '-mtune=generic' '-v'
 /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/collect2 -plugin /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/liblto_plugin.so -plugin-opt=/opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/5.2.0/lto-wrapper -plugin-opt=-fresolution=/tmp/cchfGdiM.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crti.o /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/crtbegin.o -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0 -L/opt/x86_64-linux-gnu/bin/../lib/gcc -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64 -L/opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/../../../../x86_64-linux-gnu/lib -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib -rpath-link /opt/x86_64-linux-gnu/x86_64-linux-gnu/lib64 -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/5.2.0/crtend.o /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crtn.o
/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o: In function `_start':
/workspace/srcdir/glibc-2.17/csu/../sysdeps/x86_64/start.S:119: undefined reference to `main'
collect2: error: ld returned 1 exit status

and yet we can find all the dependencies of libunwind just by setting the prefix (which I believe is used to look for the dependencies as well): https://github.com/JuliaPackaging/Yggdrasil/blob/84c1061962644addde53ef51f2ada406be61978b/L/LibUnwind/LibUnwind%401.8.1/build_tarballs.jl. LDFLAGS isn't set anywhere

[NickCao]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

[giordano]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

Not in the binarybuilder environment (which is why I'm insisting on taking it as an example).

[NickCao]

That's since libunwind mostly only depends on zlib, which is available on nearly every system.

Not in the binarybuilder environment (which is why I'm insisting on taking it as an example).

Except there is, tested with:

--- a/L/LibUnwind/LibUnwind@1.5.0/build_tarballs.jl
+++ b/L/LibUnwind/LibUnwind@1.5.0/build_tarballs.jl
@@ -29,6 +29,14 @@ if [[ ${bb_full_target} == *-sanitize+memory* ]]; then
     cp -rL ${libdir}/linux/* /opt/x86_64-linux-musl/lib/clang/*/lib/linux/
 fi

+cat > test.c <<EOF
+int main() {}
+EOF
+
+$CC -Wl,--verbose test.c -lz
+
+exit 1
+
 export CFLAGS="-DPI -fPIC"
 ./configure \
     --prefix=${prefix} \

attempt to open /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/lib/libz.so succeeded
-lz (/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/lib/libz.so

[giordano]

Except there is

That's not a libz from the sysroot (otherwise for most targets it'd be a file for a foreign architecture or OS) but it's the libz coming from the dependencies installed in the prefix (/opt/${target}/${target}/sys-root/usr/local is a symlink to ${prefix}).

That said, the libdir in the prefix (${prefix}/lib == /workspace/destdir/lib) is indeed in the linker search path:

sandbox:${WORKSPACE} # ld --verbose | grep SEARCH_DIR | tr -s ' ;' '\n'
SEARCH_DIR("=/workspace/destdir/x86_64-linux-musl/lib64")
SEARCH_DIR("=/workspace/destdir/lib64")
SEARCH_DIR("=/usr/local/lib64")
SEARCH_DIR("=/lib64")
SEARCH_DIR("=/usr/lib64")
SEARCH_DIR("=/workspace/destdir/x86_64-linux-musl/lib")
SEARCH_DIR("=/workspace/destdir/lib")
SEARCH_DIR("=/usr/local/lib")
SEARCH_DIR("=/lib")
SEARCH_DIR("=/usr/lib")

[NickCao]

That said, the libdir in the prefix (${prefix}/lib == /workspace/destdir/lib) is indeed in the linker search path:

But how does it know to add that? Is the linker configured specially or patched?