Add setup-envtest in Makefile

[darkowlzz]

Use setup-envtest for installing the binaries required for using
envtest.
The latest envtest binaries for k8s v1.21.2 are not compatible with
kubebuilder < v3, which results in envtest suite test tear down
failure. envtest fails to stop the kube-apiserver.

timeout waiting for process kube-apiserver to stop

Pin to k8s v1.19.2 envtest binaries that work with kubebuilder v2.

[hiddeco]

I think that combined with this PR we should be able to move over to the actions/envtest from fluxcd/pkg?

https://github.com/fluxcd/source-controller/blob/main/.github/workflows/e2e.yaml#L27-L30

[darkowlzz]

I've added a local github action for envtest to avoid installing go on the host for installing setup-envtest (GOROOT from the host is passed on to the run-test container and the builds in the container fails because the GOROOT passed from host doesn't exist inside the container). This will be moved to the pkg repo.
Observed some weird build failures due to the cached build files populated during setup-envtest install. Example of the build failure logs can be found here https://github.com/fluxcd/source-controller/pull/404/checks?check_run_id=3206245478#step:7:256.

Pasting some log lines below for future reference(the github actions logs will get deleted after a few months):

go: creating new go.mod: module tmp
go get: added sigs.k8s.io/controller-tools v0.5.0
cd api; /github/home/go/bin/controller-gen object:headerFile="../hack/boilerplate.go.txt" paths="./..."
-: go build internal/unsafeheader: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/32/322c1475077dbb3e3f149e186e0358a72016917dda03d9910e1abe3663d9ca04-a: no such file or directory
go build runtime/internal/atomic: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/bc/bcb9b2083df4553336339c1815cdfa18452f134067b1bd07acd1af3728d3e75c-a: no such file or directory
go build runtime/internal/sys: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/0a/0a4477cefe4c3522430b00940f0feb4ab86b0ac8c1fbb5bd17f7ff2656d5378d-a: no such file or directory
go build math/bits: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/d6/d681605cd480d72bfde50daf8de9b94205937f387746ac37eff39926be19b412-a: no such file or directory
go build unicode/utf8: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/5a/5ae44ff5b0dfd5092df2fc8641b04ad517b22012d699186e623dd84974ee1906-a: no such file or directory
go build internal/race: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/21/21204b9a9044eb2a686dfceb86034e5d753ab9105badb6f68c22358ee50bc236-a: no such file or directory
go build sync/atomic: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/2b/2ba2f289dec5650e6b5b4a6ffbc468be7e3d5aa6126a5abec1d33bc1880b2d42-a: no such file or directory
go build unicode: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/2e/2e4643207fcfbb3eb71a486abe55c0be4b140a44619554bc6d3b8413b2854609-a: no such file or directory
go build encoding: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/9b/9ba6ed76bb4a3c72b7d20f738df06c561d0b5851eb2c541c0f4a177c97bb65a8-a: no such file or directory
go build unicode/utf16: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/3a/3a210046015812087dc3ba9aa6b94b8dba3db0e9cce74fdb7426904bbab93c3c-a: no such file or directory
go build k8s.io/apimachinery/pkg/selection: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/5d/5dd2b10e90116d63cb703679741c0f45b36825521836d72a42b12ed0030a6a16-a: no such file or directory
go build github.com/google/go-cmp/cmp/internal/flags: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/8d/8d02e0eac337c439bac8d5a15b786dd122ff2b3603a03be16894144bf11cebbc-a: no such file or directory
go build internal/nettrace: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/25/255a069585d6033a295b8c4d07f3e064a1d3bbec510cfe78eb914d53523ede41-a: no such file or directory
go build internal/cpu: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/53/5354e9041207a1bfa5bc2e1f3293695eec315fd91708b5ffac90f80dcfed1594-a: no such file or directory
go build container/list: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/fd/fd12ac769da13df49f0202488b6b9962826a51d3fdf38be29efd70f440da6f94-a: no such file or directory
go build crypto/internal/subtle: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/f9/f90ccd3ff003c82b45bb3a96f1f67aef825d0a2ed33931e58cb8e447a6376424-a: no such file or directory
go build crypto/subtle: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/bb/bbe666d2773635afcfb138458060bda6ec9e79dcd9eac266247d81a24fb6da5a-a: no such file or directory
go build vendor/golang.org/x/crypto/cryptobyte/asn1: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/12/12836403574948506e35e893abf19a9e105b824030a0165b96f57bb213edbf29-a: no such file or directory
go build vendor/golang.org/x/crypto/internal/subtle: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/7b/7bc37a84c61287ab8d9481f8c770a5f0652f1adf4b0914bae8621c33500c1316-a: no such file or directory
go build k8s.io/apimachinery/pkg/types: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/b2/b2b76e52be31a92c3cc8ccf1af6b91061104f578473462bb2d245d15c0105520-a: no such file or directory
go build runtime/cgo: loading compiled Go files from cache: reading srcfiles list: cache entry not found: open /github/home/.cache/go-build/4b/4b3cf5cfa10e9b0116674cfcf5d43e9687d01359858b063815ae23d09e960345-a: no such file or directory
Error: not all generators ran successfully
run `controller-gen object:headerFile=../hack/boilerplate.go.txt paths=./... -w` to see all available markers, or `controller-gen object:headerFile=../hack/boilerplate.go.txt paths=./... -h` for usage
make: *** [Makefile:77: generate] Error 1

A workaround for this was to clean /github/home/.cache at the end of the action.

More details about the changes are in the commit messages.

[stefanprodan]

This looks great, please open a PR for updating the action in fluxcd/pkg.

Thanks @darkowlzz 🏅

[darkowlzz]

Created fluxcd/pkg#138 to update the action in fluxcd/pkg. In fluxcd/pkg, since the tests are run on host, it uncovered an issue with the KUBEBUILDER_ASSETS value, which was being set as $GITHUB_WORKSPACE/<repo-dir>/envtest. This works inside a container only. Using $RUNNER_WORKSPACE/<repo-dir>/envtest works on host and inside a container.
Once the envtest action in fluxcd/pkg is merged, I'll update this PR to use that and remove the local action.

[darkowlzz]

Updated to use fluxcd/pkg/actions/envtest 🙂

[stefanprodan]

LGTM

Thanks @darkowlzz ☕