Move to Go 1.18 (kudobuilder#354)

* Move to Go 1.18
* Contribution updates for Go 1.18, envtest and MacOs

Signed-off-by: Ken Sipe <kensipe@gmail.com>
Signed-off-by: Israel Blancas <iblancasa@gmail.com>

.circleci/config.yml

@@ -25,7 +25,7 @@ jobs:
 
   lint:
     docker:
-      - image: kudobuilder/golang:1.17
+      - image: kudobuilder/golang:1.18
     working_directory: /go/src/github.com/kudobuilder/kuttl
     steps:
       - checkout

CONTRIBUTING.md

@@ -34,11 +34,90 @@ You can find the full text of the DCO here: https://developercertificate.org/
 ### Pre-requisites
 
 - Git
-- Go `1.13` or later. Note that some [Makefile](Makefile) targets assume that your `$GOBIN` is in your `$PATH`.
+- Go `1.18` or later. Note that some [Makefile](Makefile) targets assume that your `$GOBIN` is in your `$PATH`.
 - [Kubebuilder](https://book.kubebuilder.io/quick-start.html#installation) version 2 or later - note that it is only needed for the `kube-apiserver` and `etcd` binaries, so no need to install *its* dependencies (such as `kustomize`).
-- A Kubernetes Cluster running version `1.13` or later (e.g., [kind](https://kind.sigs.k8s.io/) or [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/))
+- A Kubernetes Cluster running version `1.19` or later (e.g., [kind](https://kind.sigs.k8s.io/) or [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/))
 - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
 
+#### MacOS (darwin) and Go 1.18
+
+The `controller-gen` used with `make generate` has issues with Go 1.18 on MacOS darwin.  An example output looks like:
+
+```sh
+go: downloading golang.org/x/sys v0.0.0-20201112073958-5cba982894dd
+# golang.org/x/sys/unix
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/syscall_darwin.1_13.go:29:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.1_13.go:27:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.1_13.go:40:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:28:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:43:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:59:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:75:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:90:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:105:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:121:3: //go:linkname must refer to declared function or variable
+../../../../pkg/mod/golang.org/x/sys@v0.0.0-20201112073958-5cba982894dd/unix/zsyscall_darwin_arm64.go:121:3: too many errors
+make: *** [generate] Error 2
+```
+For the build to work you need proper version of `controller-gen` to be installed.  The current work around is:
+
+  1. change the `go.mod` L3 `go 1.18` to `go 1.17`
+  1. `make generate`
+  1. change `go.mod` back
+
+After the correct controller-gen version is installed.  All future builds will work.  The only time this work-around is necessary is if/when the controller-tools version is updated.  [The fix](https://go-review.googlesource.com/c/sys/+/274573/) has been merged into Go and should be resolved with go 1.18.1.
+
+
+#### Kubebuilder Envtest
+
+The pre-requisite for kubebuilder is for the [envtest](https://book.kubebuilder.io/reference/envtest.html).  It is used for the integration-tests, testing against kube-apiserver and etcd.  This has been complicated with Apple Silicon and process hanging etc.  The installation has changed a lot since the start of this project and is a common issue getting started.  The current use of `envtest` is to have `KUBEBUILDER_ASSETS` env variable set to the location of etcd and kube-apiserver.  For help regarding this, we found Makefile which can help.
+
+```Makefile
+.PHONY: all clean
+
+OS := $(shell uname)
+
+ifeq ($(OS),Linux)
+kernel := linux
+tar_wildcards := --wildcards
+endif
+ifeq ($(OS),Darwin)
+kernel := darwin
+tar_wildcards :=
+endif
+
+all: etcd kubectl kube-apiserver
+
+clean:
+	rm -f etcd
+	rm -f kubectl
+	rm -f kube-apiserver
+
+etcd:
+ifeq ($(OS),Linux)
+	curl -Ls https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-$(kernel)-amd64.tar.gz \
+  | tar zxv --strip-components=1 $(tar_wildcards) "*/etcd"
+endif
+ifeq ($(OS), Darwin)
+	curl -LO https://github.com/etcd-io/etcd/releases/download/v3.4.16/etcd-v3.4.16-darwin-amd64.zip
+	unzip etcd-v3.4.16-darwin-amd64.zip
+	cp etcd-v3.4.16-darwin-amd64/etcd .
+	rm etcd-v3.4.16-darwin-amd64.zip
+	rm -r etcd-v3.4.16-darwin-amd64
+endif
+
+kubectl:
+	curl -LO "https://dl.k8s.io/release/$(shell curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
+	chmod +x kubectl
+
+kube-apiserver:
+	curl -L https://storage.googleapis.com/kubebuilder-tools/kubebuilder-tools-1.19.2-$(kernel)-amd64.tar.gz \
+  | tar zxv --strip-components=2 $(tar_wildcards) "*/kube-apiserver"
+
+```
+
+Full M1 Apple Silicon support doesn't fully exist yet, and certainly not for older binaries.  This solution leverages Rosetta2. 
+
 ### Build Instructions
 
 - Get the KUTTL repo: `git clone https://github.com/kudobuilder/kuttl.git`

Makefile

@@ -86,6 +86,8 @@ generate: ## Generates code
 ifneq ($(shell go list -f '{{.Version}}' -m sigs.k8s.io/controller-tools), $(shell controller-gen --version 2>/dev/null | cut -b 10-))
 	@echo "(Re-)installing controller-gen. Current version:  $(controller-gen --version 2>/dev/null | cut -b 10-). Need $(go list -f '{{.Version}}' -m sigs.k8s.io/controller-tools)"
 	go get sigs.k8s.io/controller-tools/cmd/controller-gen@$$(go list -f '{{.Version}}' -m sigs.k8s.io/controller-tools)
+	go install sigs.k8s.io/controller-tools/cmd/controller-gen@$$(go list -f '{{.Version}}' -m sigs.k8s.io/controller-tools)
+	go mod tidy
 endif
 	controller-gen crd paths=./pkg/apis/... output:crd:dir=config/crds output:stdout
 	./hack/update_codegen.sh

go.mod

@@ -1,6 +1,6 @@
 module github.com/kudobuilder/kuttl
 
-go 1.17
+go 1.18
 
 require (
 	github.com/Masterminds/semver/v3 v3.1.0

go.sum

@@ -742,7 +742,6 @@ golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=

hack/run-e2e-tests.sh

@@ -13,7 +13,9 @@ then
     echo "Running E2E tests with junit output"
     mkdir -p reports/
     go get github.com/jstemmer/go-junit-report
-
+    go install github.com/jstemmer/go-junit-report
+    go mod tidy
+
     ./bin/kubectl-kuttl test pkg/test/test_data/ 2>&1 \
         | tee /dev/fd/2 \
         | go-junit-report -set-exit-code \

hack/run-integration-tests.sh

@@ -12,6 +12,8 @@ then
     echo "Running integration tests with junit output"
     mkdir -p reports/
     go get github.com/jstemmer/go-junit-report
+    go install github.com/jstemmer/go-junit-report
+    go mod tidy
     go test -tags integration ./pkg/... -v -mod=readonly -coverprofile cover-integration.out 2>&1 |tee /dev/fd/2 |go-junit-report -set-exit-code > reports/integration_report.xml
 else
     echo "Running integration tests without junit output"

pkg/test/utils/testing.go

@@ -5,9 +5,11 @@ import (
 	"fmt"
 	"io"
 	"os"
+	"reflect"
 	"regexp"
 	"runtime/pprof"
 	"testing"
+	"time"
 )
 
 // RunTests runs a Go test method without requiring the Go compiler.
@@ -44,7 +46,7 @@ func RunTests(testName string, testToRun string, parallelism int, testFunc func(
 			Name: testName,
 			F:    testFunc,
 		},
-	}, nil, nil).Run())
+	}, nil, nil, nil).Run())
 }
 
 // testDeps implements the testDeps interface for MainStart.
@@ -89,3 +91,40 @@ func (testDeps) StartTestLog(w io.Writer) {}
 func (testDeps) StopTestLog() error {
 	return nil
 }
+
+func (testDeps) CoordinateFuzzing(time.Duration, int64, time.Duration, int64, int, []corpusEntry, []reflect.Type, string, string) error {
+	return nil
+}
+
+func (testDeps) RunFuzzWorker(func(corpusEntry) error) error {
+	return nil
+}
+
+func (testDeps) ReadCorpus(string, []reflect.Type) ([]corpusEntry, error) {
+	return nil, nil
+}
+
+func (testDeps) CheckCorpus([]interface{}, []reflect.Type) error {
+	return nil
+}
+
+func (testDeps) ResetCoverage() {
+
+}
+
+func (testDeps) SnapshotCoverage() {
+
+}
+
+// corpusEntry is from the public go testing which references an internal structure.
+// corpusEntry is an alias to the same type as internal/fuzz.CorpusEntry.
+// We use a type alias because we don't want to export this type, and we can't
+// import internal/fuzz from testing.
+type corpusEntry = struct {
+	Parent     string
+	Path       string
+	Data       []byte
+	Values     []interface{}
+	Generation int
+	IsSeed     bool
+}

test/Dockerfile

@@ -1,4 +1,4 @@
-FROM kudobuilder/golang:1.13
+FROM kudobuilder/golang:1.18
 
 WORKDIR $GOPATH/src/github.com/kudobuilder/kuttl