makelib: introduce --enable-incremental-build, enabling "go install"

[s-urbaniak]

This change decreases incremental compilation time by 50% on my machine
for stage0 builds.

stage1 binaries are much smaller so there is no significant improvement
in compilation speed.

Small change in rkt/main.go:

$ time make
  GO           github.com/coreos/rkt/rkt
  ACTOOL       build-rkt/target/bin/stage1-coreos.aci

real    0m13.086s
user    0m11.797s
sys 0m1.323s

Small change in stage1/init/init.go:

$ time make
  GO           github.com/coreos/rkt/stage1/init
  ACTOOL       build-rkt/target/bin/stage1-coreos.aci

real    0m11.615s
user    0m10.047s
sys 0m1.083s

The same with "go build":

Small change in rkt/main.go:

$ time make
  GO           github.com/coreos/rkt/rkt
  ACTOOL       build-rkt/target/bin/stage1-coreos.aci

real    0m21.573s
user    0m32.907s
sys 0m2.177s

Small change in stage1/init/init.go:

$ time make
  GO           github.com/coreos/rkt/stage1/init
  ACTOOL       build-rkt/target/bin/stage1-coreos.aci

real    0m11.365s
user    0m10.777s
sys 0m1.083s

[lucab]

It looks like most of the workers are unhappy about the cleaning step now.

[s-urbaniak]

ack, the quickrm tool needs some massaging, since artifacts in gopath/bin and gopath/pkg are out of our control, hence we have to os.RemoveAll() for directories.

[s-urbaniak]

we might also have some issue with fedora 23, since go 1.5.3 might not yet support the -pkgdir option for non-cgo builds.

let's see what the CI is saying in the next round.

[s-urbaniak]

oh, interesting, the arm64 builders also seem to put the artifacts in a different dir.

[s-urbaniak]

nice, only arm64 needs a fix

[s-urbaniak]

arm64 places its binaries under $GOPATH/bin/linux_arm64, i.e. /home/travis/gopath/src/github.com/coreos/rkt/build-rkt-1.23.0+git/gopath/bin/linux_arm64/rkt

[s-urbaniak]

The fedora-24 failure is a known flake.

The strange thing is that this still fails on jenkins with the cross-compilation build, erroring with:

aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'

Indeed gcc-arm doesn't support -m64 according to https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html.

I guess this is an option set in a x86 build for some other binary and seems to intertwine here. I'll try to reproduce this locally.

[squeed]

AMAZING. I appreciate this beyond belief.

[s-urbaniak]

yeah, i just need to fix arm64, I think I have some silliness there left.

[s-urbaniak]

I gated this behind a ./configure --enable-incremental-build, as cross-compilation was hard (impossible?) to get right and final builds are recommended using go build anyways.

Our build system actually builds also native binaries, even when in a cross compilation scenario, i.e. depsgen is built natively amongst others. This complicates things so I suggest to keep this as least intrusive as possible.

[squeed]

Probably want to add a mention that you should not build releases with this.

[s-urbaniak]

addressed

[squeed]

A minor documentation suggestion, but LGTM.