cmd/go: cannot compile Go binary with external linking without cgo with v1.22.0

[damcadam]

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='~/.cache/go-build'
GOENV='~/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='~/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='~/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/snap/go/10506'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/snap/go/10506/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='~/bootboot/mykernel/go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4216832776=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I am trying to make BootBoot from the following path:
https://gitlab.com/bztsrc/bootboot/-/tree/master/mykernel/go?ref_type=heads

I am just running the make command.

$ make

What did you see happen?

I have found that this does not work with Go 1.22.0. The error message from making is shown below.

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

What did you expect to see?

Making this code used to work in Go 1.21.7. I expected to see the following output.

$ make
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
loadinternal: cannot find runtime/cgo
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-strip -s -K mmio -K fb -K bootboot -K environment -K initstack mykernel.aarch64.elf
aarch64-linux-gnu-readelf -hls mykernel.aarch64.elf >mykernel.aarch64.txt

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0, like so:

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

The output from making with CGO enabled while using Go 1.22.0 is also shown below.

$ go version
go version go1.22.0 linux/amd64
$ make
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# runtime/cgo
gcc_arm64.S: Assembler messages:
gcc_arm64.S:30: Error: no such instruction: `stp x29,x30,[sp,'
gcc_arm64.S:34: Error: too many memory references for `mov'
gcc_arm64.S:36: Error: no such instruction: `stp x19,x20,[sp,'
gcc_arm64.S:39: Error: no such instruction: `stp x21,x22,[sp,'
gcc_arm64.S:42: Error: no such instruction: `stp x23,x24,[sp,'
gcc_arm64.S:45: Error: no such instruction: `stp x25,x26,[sp,'
gcc_arm64.S:48: Error: no such instruction: `stp x27,x28,[sp,'
gcc_arm64.S:52: Error: too many memory references for `mov'
gcc_arm64.S:53: Error: too many memory references for `mov'
gcc_arm64.S:54: Error: too many memory references for `mov'
gcc_arm64.S:56: Error: no such instruction: `blr x20'
gcc_arm64.S:57: Error: no such instruction: `blr x19'
gcc_arm64.S:59: Error: no such instruction: `ldp x27,x28,[sp,'
gcc_arm64.S:62: Error: no such instruction: `ldp x25,x26,[sp,'
gcc_arm64.S:65: Error: no such instruction: `ldp x23,x24,[sp,'
gcc_arm64.S:68: Error: no such instruction: `ldp x21,x22,[sp,'
gcc_arm64.S:71: Error: no such instruction: `ldp x19,x20,[sp,'
gcc_arm64.S:74: Error: no such instruction: `ldp x29,x30,[sp],'
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

If I am no longer supposed to be able to use an external linker while compiling with CGO disabled, how can I get past this compilation error?

[Jorropo]

A bit of a blind shot but what happens if you use CC= to make it use gcc aarch64 ?
It could be not working because it's trying to use your host system (amd64) to build gcc_arm64.S. (but aren't theses supposed to be assembled by go's assembler ?)

[Jorropo]

I'm confused by GOARCH=arm64 GOARM=5 I'm pretty sure it's supposed to be GOARM64= for arm64 I don't think GOARM does anything here.

[seankhliao]

maybe more like #64875

[cherrymui]

External linking without cgo is fragile. The external linker will usually link in some C objects (like the startup code) anyway, and without cgo the interaction between the C code and Go code may not work correctly. So we require cgo for external linking.

To get cgo working, as @Jorropo mentioned, you'll want to set CC to the C cross compiler.

GOARM=5

It is ignored as the target is not ARM. GOARM64 does not yet exist.

-gcflags=gitlab.com/bztsrc/bootboot=-std

The -std compiler flag is for the Go standard library packages only. Please don't set this flag.

@bcmills maybe we want to mention that cgo is required for external linking in the release notes?

[bcmills]

This is working as intended — this was the fix for #46330, prior to which the linker would report loadinternal: cannot find runtime/cgo (as @damcadam reported in the Go 1.21.7 output!) but would link the binary and exit 0 despite the error.

I agree that adding a Go 1.22 release note seems appropriate.

[damcadam]

Here's the output when I run make after setting CC to the cross compiler and removing GOARM=5:

$ make
make: Warning: File 'Makefile' has modification time 244 s in the future
CGO_ENABLED=1 CC=aarch64-linux-gnu-gcc-12 GOOS=linux GOARCH=arm64 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
/snap/go/10506/pkg/tool/linux_amd64/link: running aarch64-linux-gnu-ld failed: exit status 1
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-ld: cannot find -lpthread: file format not recognized

make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

[Jorropo]

@damcadam I think you are in C cross compilation sucks land.
From experience I'm 70% sure you still have your amd64 shared objects configured in the various env variables C toolchains use to find .so files and gcc is trying to link your host amd64 pthread in the arm64 binary. It will be easier to handle that here https://go.dev/wiki/Questions

[bcmills]

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0

@damcadam, I suggest that you file an issue for that with the bootboot developers. Go does not provide a supported bare-metal port, so building a bare-metal kernel using the Go toolchain is inherently not something the Go project supports either.

See previously:

• proposal: all: add bare metal ARM support #46802
• proposal: all: add bare metal ARM support #37503