@ianlancetaylor, was the ordering of extldflags before the LDFLAGS from cgo directives in linkerFlagSupported picked intentionally? I can't tell if it's a terrible solution, but swapping the order would give the go link extdflags precedence over cgo directives for mutually exclusive linker options.

func linkerFlagSupported(arch *sys.Arch, linker, altLinker, flag string) bool {
	...
	moreFlags := trimLinkerArgv(append(flagExtldflags, ldflag...))

From here: https://cs.opensource.google/go/go/+/master:src/cmd/link/internal/ld/lib.go;l=2079;drc=1e433915ce684049a6a44fd506f691f448b56c76

I can't answer that question, but ideally we wouldn't include any absolute paths into the outputs of compilation. Maybe we could use a symlink instead of converting the sysroot to an absolute path?

That said, if flipping the order of flags is a simple way to fix this and no tests break, I'm happy to accept that change.

I don't understand what is creating the absolute path. The code you mention in cmd/cgo only applies to Go file names, not to options like --sysroot=dir.

Ah ya, misread that. Thanks.

So this is actually happening in rules_go. The flags are getting absolute here:

https://github.com/bazelbuild/rules_go/blob/aeb83e878033ef357642c87122e193df44da03fe/go/tools/builders/stdlib.go#L161-L165

So we have a --sysroot from a custom cc toolchain. When building the stdlib with go install, we make all CGO_LDFLAGS absolute, presumably because go install changes directories. If we don't, then we get an error like:

_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found

I encountered the same issue. We use a custom cc toolchain, the binaries built success but cannot run.

runtime: pcHeader: magic= 0xfffffff1 pad1= 0 pad2= 0 minLC= 1 ptrSize= 8 pcHeader.textStart= 0x1be100 text= 0x557164240100 pluginpath=
fatal error: invalid function symbol table
runtime: panic before malloc heap initialized

According @mikedanese , I try to remove the codes of absEnv as mentioned above , but not works, the issue persists.
https://github.com/bazelbuild/rules_go/blob/aeb83e878033ef357642c87122e193df44da03fe/go/tools/builders/stdlib.go#L161-L165

I found that when rule_go link the binary, it doesn't fetch clink flags through clinkopts; instead, it reads gc_linkopts and puts the flags after -extldflags into _extract_extldflags.
https://github.com/bazelbuild/rules_go/blob/c0ef535977f9fd2d9a67243552cd04da285ab629/go/private/actions/link.bzl#L64-L106

https://github.com/bazelbuild/rules_go/blob/c0ef535977f9fd2d9a67243552cd04da285ab629/go/private/actions/link.bzl#L230-L240

So, after adding the following flags to my go_binary, it started running correctly.

  gc_linkopts = [
       "-extldflags",
       "-nopie",
   ],

Wonder if this is also fixed by #4009

I believe I'm also running into this issue while trying to set up llvm with a custom sysroot. I'm finding that cgo builds (in my case a build with the race detector which seems to turn cgo on by default) in my case trigger linker errors that look like this:

ld.lld: error: relocation R_X86_64_64 cannot be used against symbol 'type:.eq.os/exec.Error'; recompile with -fPIC

After diving in fairly deep, I found that go tool link is calling my the linker with __GO_BAZEL_CC_PLACEHOLDER__ all over the place, which seems to me like perhaps an environment variable leaked out somewhere without getting substituted. The link tool runs something that looks like this to determine if linker flags are supported:

external/toolchains_llvm~~llvm~llvm_toolchain/bin/cc_wrapper.sh "-m64" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-pthread" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=__GO_BAZEL_CC_PLACEHOLDER__external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-pthread" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=__GO_BAZEL_CC_PLACEHOLDER__external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-pthread" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=__GO_BAZEL_CC_PLACEHOLDER__external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-pthread" "-fuse-ld=lld" "-Wl,--build-id=md5" "-Wl,--hash-style=gnu" "-Wl,-z,relro,-z,now" "--sysroot=__GO_BAZEL_CC_PLACEHOLDER__external/_main~_repo_rules~ubuntu_20_04_sysroot/" "-pthread" "-o" "/tmp/go-link-2282735968/a.out" "-Wl,--export-dynamic-symbol=main" "/tmp/trivial.c"

The above command fails because it can't actually find the sysroot.

So after even more digging, I've found that it seems like this string ends up embedded in the race detector somewhere. Strings on one of the race detector standard library .a files comes up with this:

[["cgo_ldflag","--target=x86_64-unknown-linux-gnu"],["cgo_ldflag","-lm"],["cgo_ldflag","-no-canonical-prefixes"],["cgo_ldflag","-fuse-ld=lld"],["cgo_ldflag","-Wl,--build-id=md5"],["cgo_ldflag","-Wl,--hash-style=gnu"],["cgo_ldflag","-Wl,-z,relro,-z,now"],["cgo_ldflag","-l:libc++.a"],["cgo_ldflag","-l:libc++abi.a"],["cgo_ldflag","-l:libunwind.a"],["cgo_ldflag","-lpthread"],["cgo_ldflag","-ldl"],["cgo_ldflag","-rtlib=compiler-rt"],["cgo_ldflag","--sysroot=__GO_BAZEL_CC_PLACEHOLDER__external/_main~_repo_rules~ubuntu_20_04_sysroot/"],["cgo_ldflag","-pthread"],["cgo_ldflag","-ldl"],["cgo_import_static","_cgo_41342379732c_Cfunc_pluginLookup"],["cgo_import_static","_cgo_41342379732c_Cfunc_pluginOpen"],["cgo_import_static","_cgo_41342379732c_Cfunc_realpath"],["cgo_import_dynamic","__libc_start_main","__libc_start_main#GLIBC_2.2.5","libc.so.6"],["cgo_import_dynamic","__gmon_start__","__gmon_start__",""],["cgo_import_dynamic","__register_frame_info","__register_frame_info",""],["cgo_import_dynamic","__cxa_finalize","__cxa_finalize#GLIBC_2.2.5","libc.so.6"],["cgo_import_dynamic","__deregister_frame_info","__deregister_frame_info",""],["cgo_import_dynamic","dlsym","dlsym#GLIBC_2.2.5","libdl.so.2"],["cgo_import_dynamic","dlerror","dlerror#GLIBC_2.2.5","libdl.so.2"],["cgo_import_dynamic","dlopen","dlopen#GLIBC_2.2.5","libdl.so.2"],["cgo_import_dynamic","realpath","realpath#GLIBC_2.3","libc.so.6"],["cgo_import_dynamic","_","_","libm.so.6"],["cgo_import_dynamic","_","_","libpthread.so.0"],["cgo_import_dynamic","_","_","libdl.so.2"],["cgo_import_dynamic","_","_","libc.so.6"]]

I only seem to have issues building programs with the race detector turned on.

Ah right, I had missed the comments before about the ordering of flags in linkerFlagsSupported. I'm indeed running into the same issue, where the valid sysroot is passing through but only at the beginning of the arg list and not the end.

I have a patch that seems to be working for my case at least and opened a PR. I don't really know if this is the right solution but at least a problem here appears to be that #4009 was only a partial fix and only handled cases for compile operations but not link operations. Unfortunately because of how go tool link works I couldn't get the exact method working and had to write a sort of hacky script-based solution instead.

Here's the upstream issue: golang/go#69954

I'll try to get to a test this week.