cmd/go: default buildmode incompatible to external linker with PIE enabled by default

[williamweixiao]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.3 linux/arm64

What operating system and processor architecture are you using (go env)?

GOARCH="arm64"
GOBIN=""
GOEXE=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/root/go-go1.7.3"
GOTOOLDIR="/root/go-go1.7.3/pkg/tool/linux_arm64"
CC="gcc"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build719384124=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

What did you do?

Following 3 steps

1. Need a GCC with PIE enabled by default
   e.g.,
   Using built-in specs.
   COLLECT_GCC=gcc
   COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-alpine-linux-musl/6.2.1/lto-wrapper
   Target: aarch64-alpine-linux-musl
   Configured with: /home/buildozer/aports/main/gcc/src/gcc-6.2.0/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=aarch64-alpine-linux-musl --host=aarch64-alpine-linux-musl --target=aarch64-alpine-linux-musl --with-pkgversion='Alpine 6.2.1' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-cloog-backend --enable-languages=c,c++,objc,java,fortran,ada --with-arch=armv8-a --with-abi=lp64 --disable-bootstrap --disable-libquadmath --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
   Thread model: posix
   gcc version 6.2.1 20160822 (Alpine 6.2.1)

2. Build Go from source code (./make.bash)

3. run go command (./go)

What did you expect to see?

run successfully

What did you see instead?

Segmentation fault

[williamweixiao]

The issue is discussed at: https://groups.google.com/forum/#!topic/golang-dev/gRCe5URKewI

[ianlancetaylor]

Looking further into this, I realize that there are still things I don't understand.

By default, the go tool is not built in external linking mode. Why is that happening for you?

When I run go build -ldflags=-v cmd/go I can see from the linker output that it is using internal linking mode, meaning that the external linker is never invoked, meaning that the default behavior of GCC is irrelevant.

By default, Go code is compiled such that it can be successfully linked into a PIE. Why is that not happening for you?

When I run go build -ldflags="-linkmode=external -extldflags=-pie -v" cmd/go I can see that the external linker is invoked, that -pie is passed to it, and that the result is a PIE. It runs correctly--it does not get a segmentation fault.

Why do your go tool crash? Can you run it under gdb and see where it goes wrong?

[mwhudson]

Something that's not made obvious enough in the report is that this is on aarch64 on alpine, so with musl as libc. But even so I don't know why this wouldn't work, Ian's points are still very relevant.

[williamweixiao]

There are three preconditions to reproduce the crash

1. ARM64 or MIPS64 platforms on which go will invoke external linker
2. Test program should use CGO (such as "go" itself) which depend on external linker
3. GCC toolchain enable PIE by default (which is the case for Alpine and other security hardened Linux distros)

I have used GDB to analyze the crash and find that it's due to following abnormal ELF relocation items:

Relocation section '.rela.dyn' at offset 0xf98 contains 67861 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000539108 000000000403 R_AARCH64_RELATIV 1fc090
000000539110 000000000403 R_AARCH64_RELATIV 1faae0

......
These relocation items are write-protections according to program headers as below:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x00000000007a7708 0x00000000007a7708 R E 10000

So the crash occurs when musl ld is trying to fill relocation address to these items at runtime
I can fix the crash if i rungo build -ldflags="-extldflags=-no-pie" cmd/go

Alpine community also has a workaround for the crash, as can be seen here: http://git.alpinelinux.org/cgit/aports/tree/community/go/default-buildmode-pie.patch
But i think the workaround can't be acceptable by other platforms since it changes default buildmode

It's interesting that X86_64 linker seems to be able to handle the "PIE" linking for "exe" buildmode. But l'd say Go needs to explicitly disable "PIE" for "exe" buildmode instead of relying on the default behavior of external linker.

So i suggest fixing the issue by turning PIE off explicitly via "-no-pie" (if recognized by GCC) when invoking external linker for "exe" buildmode.

[ianlancetaylor]

A similar thing happens on amd64--there are RELATIVE relocs against the text segment. It works nonetheless because the program has a DT_TEXTREL entry in the dynamic segment.

Can you run readelf -d on your executable to see if TEXTREL is there?

Do you know if there is something on your system that disables DT_TEXTREL processing in the dynamic linker on your system?

[ianlancetaylor]

@mwhudson @crawshaw I wonder if we should just make UseRelro always return true on an ELF system when using external linking.

[mwhudson]

Sounds like a good idea to me, I think (this stuff is fairly confusing!).

Does the new linkmode=internal buildmode=pie do the relro stuff?

[ianlancetaylor]

Yes, UseRelro returns true if Buildmode == BuildmodePIE.

[gopherbot]

CL https://golang.org/cl/33014 mentions this issue.

[ianlancetaylor]

The relro idea doesn't work, because on some platforms you have to compile with -shared in order to get an PIE that does not have a DT_TEXTREL dynamic tag.

I'd still like to understand exactly what is going wrong on Alpine GNU/Linux.