HOSTCC always uses its default linker

[nathanchance]

Currently, there is no way to specify the host linker on a kernel wide scale. HOSTLD exists in a couple of Makefiles but much like the initial premise of #342, when $(HOSTCC) is used to compile then link executables, the default ld gets invoked.

See the failure here when ld is not available in PATH: https://travis-ci.com/nathanchance/continuous-integration/jobs/194766150

We want to have completely hermetic builds where no GNU tools are invoked, meaning that we need to provide -fuse-ld= to $(HOSTCC) somehow (or have $(HOSTLD) be exclusively used for linking). We can currently do this with the variable HOSTLDFLAGS (commit) but it would be nice for HOSTLD to work like LD currently does.

[nickdesaulniers]

cc @masahir0y

[nathanchance]

Along the same lines, objtool fails to compile even when HOSTLDFLAGS=-fuse-ld=lld is provided:

error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
make: *** [Makefile:1141: prepare-objtool] Error 1

This can be fixed with the following diff:

diff --git a/Makefile b/Makefile
index 95421654a16d..f6b878084f36 100644
--- a/Makefile
+++ b/Makefile
@@ -979,7 +979,7 @@ HOST_LIBELF_LIBS = $(shell pkg-config libelf --libs 2>/dev/null || echo -lelf)
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-               echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOST_LIBELF_LIBS) -,1,0)
+               echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOSTLDFLAGS) $(HOST_LIBELF_LIBS) -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else

[nathanchance]

Recently, I found out this can be worked around by setting CLANG_DEFAULT_LINKER=lld during LLVM’s cmake’s invocation.

[tpimh]

This is what I use to set HOSTLDFLAGS:

--- a/Makefile
+++ b/Makefile
@@ -401,6 +401,9 @@ KBUILD_HOSTCFLAGS   := -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 \
 		$(HOSTCFLAGS)
 KBUILD_HOSTCXXFLAGS := -O2 $(HOST_LFS_CFLAGS) $(HOSTCXXFLAGS)
 KBUILD_HOSTLDFLAGS  := $(HOST_LFS_LDFLAGS) $(HOSTLDFLAGS)
+ifeq ($(shell $(HOSTLD) -v | grep -o ^LLD),LLD)
+KBUILD_HOSTLDFLAGS += -fuse-ld=lld
+endif
 KBUILD_HOSTLDLIBS   := $(HOST_LFS_LIBS) $(HOSTLDLIBS)
 
 # Make variables (CC, etc...)
@@ -996,7 +999,7 @@ HOST_LIBELF_LIBS = $(shell pkg-config libelf --libs 2>/dev/null || echo -lelf)
 
 ifdef CONFIG_STACK_VALIDATION
   has_libelf := $(call try-run,\
-		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(HOST_LIBELF_LIBS) -,1,0)
+		echo "int main() {}" | $(HOSTCC) -xc -o /dev/null $(KBUILD_HOSTLDFLAGS) $(HOST_LIBELF_LIBS) -,1,0)
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else

Still not perfect, would fail if lld is not found in path and HOSTLD variable contains absolute path to it. Using $(which ${LD}) would solve it, but would also fail in other uses.

[nickdesaulniers]

I wonder what hack I put in pixel 4 kernel?
https://android.googlesource.com/kernel/msm/+/refs/heads/android-msm-coral-4.14-android10-c2f2/Makefile#508

Yeah, I should probably revisit upstreaming that series; IIRC I did send that as a patch.

[nickdesaulniers]

oh, nvm, this bug is about HOSTCC

[nickdesaulniers]

https://lore.kernel.org/lkml/20200820220955.3325555-1-ndesaulniers@google.com/

[nathanchance]

This is still relevant.

Looks like @nickdesaulniers worked around this in Android with https://android.googlesource.com/kernel/common/+/537c1df08e0bc4d9e7d139539d85343f88db0bf8 because Masahiro would not accept it.

I think the path of least resistance forward is sending my diff above that adds $(HOSTLDFLAGS) or $(KBUILD_HOSTLDFLAGS) to finding libelf then recommending that anyone who wants to have a fully LLVM build just build their kernel with HOSTLDFLAGS=-fuse-ld=lld so I have done just that: https://lore.kernel.org/r/20210422201914.3682494-1-nathan@kernel.org/

Setup:

$ mkdir bin

$ for binary in ar as g++ gcc ld nm objcopy objdump readelf size string; do ln -fsv /bin/false bin/$binary; done
'bin/ar' -> '/bin/false'
'bin/as' -> '/bin/false'
'bin/g++' -> '/bin/false'
'bin/gcc' -> '/bin/false'
'bin/ld' -> '/bin/false'
'bin/nm' -> '/bin/false'
'bin/objcopy' -> '/bin/false'
'bin/objdump' -> '/bin/false'
'bin/readelf' -> '/bin/false'
'bin/size' -> '/bin/false'
'bin/string' -> '/bin/false'

Prepatch:

$ PATH=$PWD/bin:$PATH make -skj"$(nproc)" HOSTLDFLAGS=-fuse-ld=lld LLVM=1 LLVM_IAS=1 distclean defconfig all
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
make[1]: *** [Makefile:1260: prepare-objtool] Error 1

Postpatch:

$ PATH=$PWD/bin:$PATH make -skj"$(nproc)" HOSTLDFLAGS=-fuse-ld=lld LLVM=1 LLVM_IAS=1 distclean defconfig all

$ echo $?
0

If that patch is accepted, Android could drop that out of tree patch then use HOSTLDFLAGS in their build script.

[nathanchance]

Patch accepted: https://git.kernel.org/masahiroy/linux-kbuild/c/f634ca650f724347892068489c7920631a3aac6a

[nathanchance]

Merged into mainline: https://git.kernel.org/torvalds/c/f634ca650f72

My recommendation would be that Android takes advantage of this variable on the command line via build.sh and revert

https://android.googlesource.com/kernel/common/+/537c1df08e0bc4d9e7d139539d85343f88db0bf8

https://android.googlesource.com/kernel/common/+/f14e5bfb39b776318133e00b8c066ce8a870e9ab

but that is up to them :) closing this for now.