investigate speeding up CI builds

[rdallman]

• we pull a lot of images
• it's not immediately clear what's slow because we use shell scripts for the tests and CI only reports the whole script as a stage (i.e. break these up to debug)
• master builds some images, maybe that is slow too?
• see what else may take a while and speed it up!

[skinowski]

Recent addition of fn-test-utils image increased this significantly. Looking into it.

[skinowski]

#619 related

[skinowski]

Extensions and middleware compilations at the end of test.sh script is taking forever since 'go build' goes seem to perform incremental builds.

[skinowski]

Identified the real culprit in this, looks like fn-test-utils container stays in a busy-loop spewing errors. See:

fnproject/fdk-go#8

This causes rest of the build to take forever.

[skinowski]

Also a poor man's attempt to speed up a few things in test.sh and release.sh:

#620

[skinowski]

Container leak workaround in runner_test.go:

#622

[denismakogon]

make docker-build, we use vanilla go 1.9 alpine image and we do install there few more packages, so could someone tell if we can create our own version of go 1.9 alpine with all of that packages and just use that as the base image for build fnserver image? It takes a lot of time in CI.

[denismakogon]

swagger image is damn huge for just one binary file, almost half a gig. Maybe we can just pull binary release from here https://github.com/go-swagger/go-swagger/releases and use that binary instead whole image?

[denismakogon]

Middleware and extensions: CI builds at least 2 different binaries for those to see if API is not broken. So, just specifically for testing, can’t we go with 1 example that includes both middleware and extension?

[denismakogon]

Datastore images: mysql, redis, postgres, minio are huge, we need to get smaller (alpine?) images for testing in CI.

[skinowski]

Played around with #625 an attempt to reduce shell script stuff into Makefile itself. Brings the build down to 14 min.

[rdallman]

down to 5 minutes now. awesome, thanks!

some more ideas:

#629 (comment)
#629 (comment)

[rdallman]

4 minutes and change now.

the go install caching, if possible, should save about another minute off. would be sweet, but this is pretty bearable for now. we could leave this open I guess as ongoing but seems like we're in a pretty good spot