cmd/go: cannot compile Go binary with external linking without cgo with v1.22.0

[damcadam]

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='~/.cache/go-build'
GOENV='~/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='~/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='~/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/snap/go/10506'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/snap/go/10506/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='~/bootboot/mykernel/go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4216832776=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I am trying to make BootBoot from the following path:
https://gitlab.com/bztsrc/bootboot/-/tree/master/mykernel/go?ref_type=heads

I am just running the make command.

$ make

What did you see happen?

I have found that this does not work with Go 1.22.0. The error message from making is shown below.

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

What did you expect to see?

Making this code used to work in Go 1.21.7. I expected to see the following output.

$ make
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
loadinternal: cannot find runtime/cgo
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-strip -s -K mmio -K fb -K bootboot -K environment -K initstack mykernel.aarch64.elf
aarch64-linux-gnu-readelf -hls mykernel.aarch64.elf >mykernel.aarch64.txt

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0, like so:

CGO_ENABLED=0 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
-linkmode requires external (cgo) linking, but cgo is not enabled
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

The output from making with CGO enabled while using Go 1.22.0 is also shown below.

$ go version
go version go1.22.0 linux/amd64
$ make
CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOARM=5 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# runtime/cgo
gcc_arm64.S: Assembler messages:
gcc_arm64.S:30: Error: no such instruction: `stp x29,x30,[sp,'
gcc_arm64.S:34: Error: too many memory references for `mov'
gcc_arm64.S:36: Error: no such instruction: `stp x19,x20,[sp,'
gcc_arm64.S:39: Error: no such instruction: `stp x21,x22,[sp,'
gcc_arm64.S:42: Error: no such instruction: `stp x23,x24,[sp,'
gcc_arm64.S:45: Error: no such instruction: `stp x25,x26,[sp,'
gcc_arm64.S:48: Error: no such instruction: `stp x27,x28,[sp,'
gcc_arm64.S:52: Error: too many memory references for `mov'
gcc_arm64.S:53: Error: too many memory references for `mov'
gcc_arm64.S:54: Error: too many memory references for `mov'
gcc_arm64.S:56: Error: no such instruction: `blr x20'
gcc_arm64.S:57: Error: no such instruction: `blr x19'
gcc_arm64.S:59: Error: no such instruction: `ldp x27,x28,[sp,'
gcc_arm64.S:62: Error: no such instruction: `ldp x25,x26,[sp,'
gcc_arm64.S:65: Error: no such instruction: `ldp x23,x24,[sp,'
gcc_arm64.S:68: Error: no such instruction: `ldp x21,x22,[sp,'
gcc_arm64.S:71: Error: no such instruction: `ldp x19,x20,[sp,'
gcc_arm64.S:74: Error: no such instruction: `ldp x29,x30,[sp],'
make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

If I am no longer supposed to be able to use an external linker while compiling with CGO disabled, how can I get past this compilation error?

[Jorropo]

A bit of a blind shot but what happens if you use CC= to make it use gcc aarch64 ?
It could be not working because it's trying to use your host system (amd64) to build gcc_arm64.S. (but aren't theses supposed to be assembled by go's assembler ?)

[Jorropo]

I'm confused by GOARCH=arm64 GOARM=5 I'm pretty sure it's supposed to be GOARM64= for arm64 I don't think GOARM does anything here.

[seankhliao]

maybe more like #64875

[cherrymui]

External linking without cgo is fragile. The external linker will usually link in some C objects (like the startup code) anyway, and without cgo the interaction between the C code and Go code may not work correctly. So we require cgo for external linking.

To get cgo working, as @Jorropo mentioned, you'll want to set CC to the C cross compiler.

GOARM=5

It is ignored as the target is not ARM. GOARM64 does not yet exist.

-gcflags=gitlab.com/bztsrc/bootboot=-std

The -std compiler flag is for the Go standard library packages only. Please don't set this flag.

@bcmills maybe we want to mention that cgo is required for external linking in the release notes?

[bcmills]

This is working as intended — this was the fix for #46330, prior to which the linker would report loadinternal: cannot find runtime/cgo (as @damcadam reported in the Go 1.21.7 output!) but would link the binary and exit 0 despite the error.

I agree that adding a Go 1.22 release note seems appropriate.

[damcadam]

Here's the output when I run make after setting CC to the cross compiler and removing GOARM=5:

$ make
make: Warning: File 'Makefile' has modification time 244 s in the future
CGO_ENABLED=1 CC=aarch64-linux-gnu-gcc-12 GOOS=linux GOARCH=arm64 go build -trimpath -gcflags=gitlab.com/bztsrc/bootboot=-std -ldflags="-linkmode external -extld aarch64-linux-gnu-ld -extldflags '-nostdlib -n -v -static -m aarch64elf -T link.ld'" -o mykernel.aarch64.elf
# gitlab.com/bztsrc/bootboot
/snap/go/10506/pkg/tool/linux_amd64/link: running aarch64-linux-gnu-ld failed: exit status 1
GNU ld (GNU Binutils for Ubuntu) 2.38
aarch64-linux-gnu-ld: cannot find -lpthread: file format not recognized

make: *** [Makefile:47: mykernel.aarch64.elf] Error 1

[Jorropo]

@damcadam I think you are in C cross compilation sucks land.
From experience I'm 70% sure you still have your amd64 shared objects configured in the various env variables C toolchains use to find .so files and gcc is trying to link your host amd64 pthread in the arm64 binary. It will be easier to handle that here https://go.dev/wiki/Questions

[bcmills]

I am developing an embedded OS in Go. Compiling this code (the default bootboot kernel) works with Go version 1.21.7, but fails with the compiler changes in Go 1.22.0

@damcadam, I suggest that you file an issue for that with the bootboot developers. Go does not provide a supported bare-metal port, so building a bare-metal kernel using the Go toolchain is inherently not something the Go project supports either.

See previously:

• proposal: all: add bare metal ARM support #46802
• proposal: all: add bare metal ARM support #37503

[damcadam]

Hello, it looks like someone filed an issue with BootBoot that's having the same problem. Here's the link to that:
https://gitlab.com/bztsrc/bootboot/-/issues/87#note_1773739397
The BootBoot developers suggested checking the release log of Go 1.22.0 to see what has changed since Go 1.21.6. The person was also directed to ask on Go forums instead of BootBoot.

[gopherbot]

Change https://go.dev/cl/569296 mentions this issue: _content/doc: mention cgo requirements for external linking in release notes for Go 1.21 and 1.22

[bcmills]

@damcadam, I'm sorry, but what you're asking for just isn't a supported Go build configuration. Perhaps you can change the BootBoot Go example to build using internal linking instead of external?

[damcadam]

@bcmills If building with internal linking will work for Go 1.22 and future versions of Go that would be awesome! Do you have any guidance or resources I could use for using internal linking instead of external linking?

[cherrymui]

Passing -ldflags=-linkmode=internal will make it use internal linking mode. But it doesn't support linker script. What do you need specifically with the linker script?

[damcadam]

Sorry about the late reply. I ended up using the -buildmode=c-archive flag for go build and linking all my .o files with the archive file. This works for me, and I don't have any problems. Thanks for considering the issue and for the help!