cmd/go: 'go build' does not cache linked executables

[dan-lind]

What version of Go are you using (go version)?

$ go version
 go1.12.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ko1dli/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ko1dli/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/mnt/c/Users/ko1dli/git/serverless-alias-search/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build698955985=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ time go build ./...
go build ./...  55.27s user 23.34s system 278% cpu 28.186 total
$ time go build ./...
go build ./...  57.38s user 22.05s system 275% cpu 28.872 total
$ time go install ./...
go install ./...  56.70s user 24.88s system 274% cpu 29.669 total
$ time go install ./...
go install ./...  0.70s user 2.67s system 168% cpu 2.003 total

What did you expect to see?

I expected go build to do an incremental build, but it seems like it recompiles everything every time I build. See go install for comparison

What did you see instead?

I expected the second go build to be incremental, rather than rebuilding everything again.

[katiehockman]

/cc @bcmills @jayconrod

[jayconrod]

go build should be building incrementally.

Can you give us more info to reproduce this? Specifically, we'd be looking for a test repository that reproduces this problem and output of two consecutive runs of go build -x ./... that shows the same commands being executed.

[dan-lind]

I've added the output of three consecutive time go build -x ./... after first doing go clean
The first one obviously takes longer because of some downloads, but the seconds and third ones are identical from what I can see. Is this helpful by itself?

I will look into providing a test repo, but I reckon it might take some time since the code is not open source.

build1.txt
build2.txt
build3.txt

[jayconrod]

It looks like all the actions in the second and third logs are link actions. That makes sense I guess. Executables never get written to the cache, but when executables are installed (with go install, go get, go build -i), the link may be skipped for target executables that are up to date.

Executables were originally not written to the cache because before 1.12, caching was optional, and there was nowhere to put them outside of the cache if we weren't installing them.

Now that the cache is mandatory, we could store executables in the cache fairly easily. This may increase cache size significantly though, and it doesn't seem like there's much benefit.

[dan-lind]

@jayconrod You say that executables never get written to the cache, and mention go build -i which makes me a bit puzzled. The Go 1.10 release notes says

The go build command now maintains a cache of recently built packages, separate from the installed packages in $GOROOT/pkg or $GOPATH/pkg. The effect of the cache should be to speed builds that do not explicitly install packages or when switching between different copies of source code (for example, when changing back and forth between different branches in a version control system). The old advice to add the -i flag for speed, as in go build -i or go test -i, is no longer necessary: builds run just as fast without -i. For more details, see go help cache.

As I understand it they should be written to the cache, although a separate one, even without using the -i option?

[jayconrod]

I think that snippet is referring to packages (.a files) only. I don't see mention of binaries.

When building with go install, we do treat installed binaries as part of the cache, even though they aren't in the cache directory. That's why you saw the speedup earlier on the second go install command.

[jeffsaremi]

I'm experiencing the exact same thing. Never been able to do any incremental builds. Go build or even Go install will build the whole thing every time

go version go1.9.3 linux/amd64

I can give you the complete project to investigate once you let me know how I can upload files for you

[bcmills]

@jeffsaremi, per the release policy Go 1.9 has not been supported since 2018-08-24 (when Go 1.11 was released). Please upgrade to a supported version (1.12 or 1.13) and see whether you can still reproduce the issue.

[jeffsaremi]

Thank you. I shall do as instructed and report back

[jeffsaremi]

As per @bcmills recommendation, I upgraded go and the problem is solved. go build now does incremental build with beautifully
thanks

go version go1.13 linux/amd64

[dan-lind]

@jeffsaremi I guess you did go install then?
With go build I'm still seeing the same behaviour as before with 1.13, which is what I expect since this issue is still open :-)

[jeffsaremi]

No I did a plain go build. It was smart enough not to rebuild the final binary.