Skip to content

cmd/go: builds not reproducible #36230

Closed
Closed
@jakecraige

Description

@jakecraige

What version of Go are you using (go version)?

$ go version
go version go1.13.5 darwin/amd64
$ docker run --rm golang:1.13.5 go version
go version go1.13.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/username/Library/Caches/go-build"
GOENV="/Users/username/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/username/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/nz/pfhjh4nj1zx5czg6r89fxpyh0000gp/T/go-build775226103=/tmp/go-build -gno-record-gcc-switches -fno-common"
docker run --rm golang:1.13.5 go env Output
$ docker run --rm golang:1.13.5 go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build001072247=/tmp/go-build -gno-record-gcc-switches"

What did you do?

/*
* reproducible.go
*
* This file shows some unexpected differences in go binaries when we expect to have byte-for-byte
* deterministic builds. This seems to surface when the same binary is built on two different
* machines. We simulate that in this example by using docker for the second.
*
* 1. Build the first binary normally, explicitly setting options for build reproducibility:
*    `GOOS=darwin GOARCH=amd64 go build -trimpath -ldflags="-buildid=" -o one reproducible.go`
* 2. Build the second in a docker container
*    `docker run --rm -v $(pwd):/app -w /app -e GOOS=darwin -e GOARCH=amd64 golang:1.13.5 go build -trimpath -ldflags="-buildid=" -o two reproducible.go`
* 3. Shasum files for comparison: `shasum one two` to see they don't match. Analyze deper with tools like `vbindiff one two`
 */
package main

import "fmt"

func main() {
	fmt.Println("Hello world")
}

What did you expect to see?

The shasum of binaries one and two should be equal.

What did you see instead?

The shasums do not match

eb264107f7305f70fa4fb22be0c7b7db18fa3e69  one
0b5fd4561dd4a21aad2d91ef28a73cc42feff5c9  two

I know that building on a Mac vs Linux may have inconsistencies due to the environment and if I had an easy way to simulate that I would have for this issue. This is the reduction of seeing the same issue on a more complex project where we did see that the binaries were different in a similar way when one Mac built it in docker vs another Mac that built it in docker. I would expect that if a reviewer of this issue does the docker build they get the same output as me, but when another on my team did (not for this exact example unfortunately) they did not.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions