Skip to content

Commit

Permalink
Support Helm 3 (#1811)
Browse files Browse the repository at this point in the history 
* Getting there, saving
* Dep ensure: exit 0!
* Bump solo-kit
* Go get dependencies ignored by dep
* Convert to Helm 3
* Remove namespace.create from Helm value file templates
* Add Helm hook-delete-policy to gateway certgen job
* Make webhook configuration a Helm post-delete hook
* Remove unnecessary Helm hooks from gloo CRs
* Remove all unnecessary Helm hooks
* Consolidate certgen job resources

* Move them to a single file
* Do proper cleanup
* Execute hooks also during upgrade
* Fix k8s 1.16 compile error
* Got dep happy again after importing helm.sh/helm/v3/pkg/action
* Reluctantly turn gloo CRs back into hooks
* Fix go.mod
* Got simple install via Helm 3 libs working
* Update CI from Helm 2.16.1 to latest Helm version (3.0.0) (#1791)

* Update CI from Helm 2.16.1 to latest Helm version (3.0.0)
* Update go-make and e2e-ginkgo in cloudbuild.yaml to pick up Helm cloud-builders change.
* Remove installation ID labels
* Mark hook cleanup resources with a annotation
* Add constant with Helm hook cleanup annotation name
* Installation unit tests are now passing
* Merge branch 'helm-3-dep-unit-tests' into helm-3-dep
* Add back namespace template
* Package a single, reusable value file with the chart
* Re-implement `glooctl install` commands
* Re-implement `glooctl uninstall` command
* Remove installationId from templates
* Update Helm README.md
* Update Helm make targets
* Fix override value generation for ingress/knative modes
* Add constant with Helm hook cleanup annotation name
* Implement dry-run
* Remove unused files
* Remove leftover installationIds
* Dep ensure
* Start getting codegen to work
* Fix go mod dependency on ilackarms/protokit
* Codegen
* Merge branch 'feature-rc1' into helm-3-dep

# Conflicts:
#	Gopkg.lock
#	docs/content/advanced_configuration/fds_mode.md
#	install/test/helm_test.go
* Dep ensure
* Codegen
* Wire up --verbose flag to install
* Add option to wire up uninstall --verbose flag, even though it is currently ignored.
* Add verbose logging for install
* Changelog
* Format code
* Fix check-code-and-docs-gen.sh script
* GO111MODULE=on for all `go run` and `go build` commands
* Fix typo in changelog
* Install/uninstall unit tests (#1816)

* Install tests in progress
* Working on install unit tests
* Install/uninstall tests passing
* fmt
* Fix kubectl editor
* Remove legacy imports
* Use knative/serving 0.10.0 (#1819)

* Use knative/serving 0.10.0
* Re-add GO111MODULES=on to ingress build
* Fix helm test regression
* Make sure helm unit tests dont reach out to the k8s api server
* Remove dead comment
* Bump reporting-client to 0.1.2
* GO111MODULE=on on oss compliance check
* Use go 1.12 image for check-code-and-docs-gen cloudbuild step
* Disable licence info generation
* Pin stackdriver dependency
* Add temp log line to first suite

Want to see if the long test startup time is actually due to slow compilation
* Try k8s 1.15, should look to 1.16.2
* Try k8s 1.16.2
* Consul to 1.5.2
* Add GO_BUILD_FLAGS in makefile
* changelog
* Merge branch 'helm-3-dep' of https://github.com/solo-io/gloo into helm-3-dep
* Add back the --all and --delete-namespace flags to `glooctl uninstall`
* Create namespace as part of regressions test BeforeSuite
* Fix go.mod vault dependency
* Pin cespare/xxhash/v2 in go.mod file
* Verbose logging on install - output helm overrides as yaml
* Deal with xxhash/v2 special dependency
* changelog
* Update docs/content/advanced_configuration/fds_mode.md

Co-Authored-By: Graham Goudeau <grahamgoudeau@gmail.com>
* Remove legacy imports
* Use knative version 0.10.0
* Add --release-name flag to install and uninstall commands (#1826)

* Add --release-name flag to install and uninstall commands

* fix test

* Add --release-name to install and uninstall commands
* Remove depreciated code
* use passed-in releaseName instead of constants.GlooReleaseName
* Update dep to use new solo-kit release
* Formatting
* Regen code
* Ensure no rbac resources are created if .Values.global.glooRbac.create=false
* Fix rbac helm setting
* Fix go mod after solo-kit release
* Knative install fix
* Knative install fix, command needs to be passed as an array
* Remove println
* Fix consul unit test
* Remove `go-get-ignored-deps` make target
* Try and add back third party licence generation script
* Revert "Try and add back third party licence generation script"

This reverts commit a4217d1
* Addressed outstanding PR comments
* Refactored helm actions file
* Addressed further CR comments
* Codegen
  • Loading branch information
@marcogschmidt @soloio-bulldozer
marcogschmidt authored and soloio-bulldozer[bot] committed Dec 6, 2019
1 parent f2530c3 commit fc5c370
Show file tree
Hide file tree
Showing 119 changed files with 7,239 additions and 4,158 deletions.
1,612 changes: 988 additions & 624 deletions Gopkg.lock
View file

Large diffs are not rendered by default.

82 changes: 50 additions & 32 deletions Gopkg.toml
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
# This file remains temporarily because several libaries parse
# toml files and our cloudbuild setup depends on GOPATH.
#
# Gloo is currently building using go modules, however. We are
# actively working to rewrite our parsing libraries to use go
# modules and updating our CI to work outside of the GOPATH.

ignored = [
"github.com/Masterminds/semver/v3",
"github.com/Masterminds/sprig/v3",
"github.com/pierrec/lz4/v3",
"github.com/russross/blackfriday/v2",
"github.com/cespare/xxhash/v2",
]

[prune]
go-tests = true
unused-packages = true
Expand All @@ -14,10 +29,6 @@
version = "1.3.0"
name = "github.com/gogo/protobuf"

[[override]]
branch = "master"
name = "k8s.io/api"

[[override]]
version = "1.3.1"
name = "github.com/golang/protobuf"
Expand All @@ -28,19 +39,28 @@

[[constraint]]
name = "github.com/solo-io/solo-kit"
version = "0.11.12"
version = "0.11.13"

[[override]]
name = "github.com/evanphx/json-patch"
version = "3.0.0"

[[override]]
name = "helm.sh/helm/v3"
version = "3.0.0"
source = "https://github.com/helm/helm"

[[override]]
name = "github.com/solo-io/go-utils"
version = "0.10.25"
version = "0.11.0"

[[constraint]]
name = "github.com/hashicorp/consul"
version = "1.5.2"

[[constraint]]
name = "github.com/solo-io/reporting-client"
version = "0.1.1"
version = "0.1.2"

[[constraint]]
name = "github.com/onsi/gomega"
Expand All @@ -55,54 +75,52 @@

[[override]]
name = "k8s.io/kubernetes"
version = "=v1.13.1"
version = "v1.16.2"

[[constraint]]
[[override]]
name = "k8s.io/api"
version = "kubernetes-1.13.1"
version = "kubernetes-1.16.2"

[[constraint]]
name = "k8s.io/kubectl"
version = "kubernetes-1.16.2"

[[override]]
name = "k8s.io/client-go"
version = "kubernetes-1.13.1"
version = "kubernetes-1.16.2"

[[override]]
name = "k8s.io/apimachinery"
version = "kubernetes-1.13.1"
version = "kubernetes-1.16.2"

[[override]]
name = "k8s.io/apiextensions-apiserver"
version = "kubernetes-1.13.1"
version = "kubernetes-1.16.2"

[[override]]
name = "k8s.io/apiserver"
version = "kubernetes-1.13.1"
version = "kubernetes-1.16.2"

[[override]]
name = "k8s.io/helm"
version = "v2.12.3"
name = "k8s.io/klog"
revision = "9cbb78b20423182f9e5b2a214dd255f5e117d2d1"
source = "github.com/stefanprodan/klog"

[[override]]
name = "github.com/go-openapi/spec"
version = "0.19.2"
name = "k8s.io/utils"
revision = "6ca3b61696b65b0e81f1a39b4937fc2d2994ed6a"

[[constraint]]
name = "knative.dev/serving"
version = "v0.8.0"

# added to support knative v0.8.0
[[override]]
name = "golang.org/x/net"
revision = "aaf60122140d3fcf75376d319f0554393160eb50"
name = "github.com/go-openapi/spec"
version = "0.19.2"

# added to support knative v0.8.0
[[override]]
name = "golang.org/x/xerrors"
revision = "a985d3407aa71f30cf86696ee0a2f409709f22e1"
name = "github.com/docker/docker"
revision = "be7ac8be2ae072032a4005e8f232be3fc57e4127"

[[override]]
name = "k8s.io/klog"
revision = "9cbb78b20423182f9e5b2a214dd255f5e117d2d1"
source = "github.com/stefanprodan/klog"
[[constraint]]
name = "knative.dev/serving"
version = "v0.10.0"

[[constraint]]
name = "github.com/spf13/viper"
Expand Down
55 changes: 30 additions & 25 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ GLOOE_VERSION ?= 0.20.4
LDFLAGS := "-X github.com/solo-io/gloo/pkg/version.Version=$(VERSION) -X github.com/solo-io/gloo/pkg/version.EnterpriseTag=$(GLOOE_VERSION)"
GCFLAGS := all="-N -l"

GO_BUILD_FLAGS := GO111MODULE=on CGO_ENABLED=0 GOARCH=amd64

# Passed by cloudbuild
GCLOUD_PROJECT_ID := $(GCLOUD_PROJECT_ID)
BUILD_ID := $(BUILD_ID)
Expand Down Expand Up @@ -73,7 +75,7 @@ update-deps:

.PHONY: pin-repos
pin-repos:
go run pin_repos.go
GO111MODULE=on go run pin_repos.go

.PHONY: check-format
check-format:
Expand Down Expand Up @@ -108,8 +110,8 @@ generated-code: $(OUTPUT_DIR)/.generated-code verify-enterprise-protos update-li
SUBDIRS:=$(shell ls -d -- */ | grep -v vendor)
$(OUTPUT_DIR)/.generated-code:
find . -name *.sk.md | xargs rm
rm docs/content/cli/glooctl*; go run projects/gloo/cli/cmd/docs/main.go
go generate ./...
rm docs/content/cli/glooctl*; GO111MODULE=on go run projects/gloo/cli/cmd/docs/main.go
GO111MODULE=on go generate ./...
gofmt -w $(SUBDIRS)
goimports -w $(SUBDIRS)
mkdir -p $(OUTPUT_DIR)
Expand All @@ -119,7 +121,7 @@ $(OUTPUT_DIR)/.generated-code:
.PHONY: verify-enterprise-protos
verify-enterprise-protos:
@echo Verifying validity of generated enterprise files...
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build projects/gloo/pkg/api/v1/enterprise/verify.go
$(GO_BUILD_FLAGS) GOOS=linux go build projects/gloo/pkg/api/v1/enterprise/verify.go

#----------------------------------------------------------------------------------
# Generate mocks
Expand Down Expand Up @@ -159,16 +161,16 @@ generate-client-mocks:
CLI_DIR=projects/gloo/cli

$(OUTPUT_DIR)/glooctl: $(SOURCES)
go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go
GO111MODULE=on go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go

$(OUTPUT_DIR)/glooctl-linux-amd64: $(SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go

$(OUTPUT_DIR)/glooctl-darwin-amd64: $(SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=darwin go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=darwin go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go

$(OUTPUT_DIR)/glooctl-windows-amd64.exe: $(SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=windows go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=windows go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CLI_DIR)/cmd/main.go


.PHONY: glooctl
Expand All @@ -191,7 +193,7 @@ GATEWAY_DIR=projects/gateway
GATEWAY_SOURCES=$(call get_sources,$(GATEWAY_DIR))

$(OUTPUT_DIR)/gateway-linux-amd64: $(GATEWAY_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(GATEWAY_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(GATEWAY_DIR)/cmd/main.go


.PHONY: gateway
Expand All @@ -213,7 +215,7 @@ INGRESS_DIR=projects/ingress
INGRESS_SOURCES=$(call get_sources,$(INGRESS_DIR))

$(OUTPUT_DIR)/ingress-linux-amd64: $(INGRESS_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(INGRESS_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(INGRESS_DIR)/cmd/main.go


.PHONY: ingress
Expand All @@ -235,7 +237,7 @@ ACCESS_LOG_DIR=projects/accesslogger
ACCESS_LOG_SOURCES=$(call get_sources,$(ACCESS_LOG_DIR))

$(OUTPUT_DIR)/access-logger-linux-amd64: $(ACCESS_LOG_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(ACCESS_LOG_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(ACCESS_LOG_DIR)/cmd/main.go


.PHONY: access-logger
Expand All @@ -257,7 +259,7 @@ DISCOVERY_DIR=projects/discovery
DISCOVERY_SOURCES=$(call get_sources,$(DISCOVERY_DIR))

$(OUTPUT_DIR)/discovery-linux-amd64: $(DISCOVERY_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(DISCOVERY_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(DISCOVERY_DIR)/cmd/main.go


.PHONY: discovery
Expand All @@ -279,7 +281,7 @@ GLOO_DIR=projects/gloo
GLOO_SOURCES=$(call get_sources,$(GLOO_DIR))

$(OUTPUT_DIR)/gloo-linux-amd64: $(GLOO_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(GLOO_DIR)/cmd/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(GLOO_DIR)/cmd/main.go


.PHONY: gloo
Expand All @@ -302,7 +304,7 @@ ENVOYINIT_DIR=projects/envoyinit/cmd
ENVOYINIT_SOURCES=$(call get_sources,$(ENVOYINIT_DIR))

$(OUTPUT_DIR)/envoyinit-linux-amd64: $(ENVOYINIT_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(ENVOYINIT_DIR)/main.go
$(GO_BUILD_FLAGS) GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(ENVOYINIT_DIR)/main.go

.PHONY: envoyinit
envoyinit: $(OUTPUT_DIR)/envoyinit-linux-amd64
Expand All @@ -326,7 +328,7 @@ CERTGEN_DIR=jobs/certgen/cmd
CERTGEN_SOURCES=$(call get_sources,$(CERTGEN_DIR))

$(OUTPUT_DIR)/certgen-linux-amd64: $(CERTGEN_SOURCES)
CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CERTGEN_DIR)/main.go
GO111MODULE=on CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags=$(LDFLAGS) -gcflags=$(GCFLAGS) -o $@ $(CERTGEN_DIR)/main.go

.PHONY: certgen
certgen: $(OUTPUT_DIR)/certgen-linux-amd64
Expand Down Expand Up @@ -357,14 +359,14 @@ HELM_DIR := install/helm
INSTALL_NAMESPACE ?= gloo-system

.PHONY: manifest
manifest: prepare-helm install/gloo-gateway.yaml install/gloo-knative.yaml update-helm-chart
manifest: prepare-helm install/gloo-gateway.yaml install/gloo-ingress.yaml install/gloo-knative.yaml update-helm-chart

# creates Chart.yaml, values.yaml, values-knative.yaml, values-ingress.yaml. See install/helm/gloo/README.md for more info.
# Creates Chart.yaml and values.yaml. See install/helm/gloo/README.md for more info.
.PHONY: prepare-helm
prepare-helm: $(OUTPUT_DIR)/.helm-prepared

$(OUTPUT_DIR)/.helm-prepared:
go run install/helm/gloo/generate.go $(VERSION)
GO111MODULE=on go run install/helm/gloo/generate.go $(VERSION)
touch $@

update-helm-chart:
Expand All @@ -383,10 +385,12 @@ install/gloo-gateway.yaml: prepare-helm
helm template install/helm/gloo $(HELMFLAGS) | tee $@ $(OUTPUT_YAML) $(MANIFEST_OUTPUT)

install/gloo-knative.yaml: prepare-helm
helm template install/helm/gloo $(HELMFLAGS) --values install/helm/gloo/values-knative.yaml | tee $@ $(OUTPUT_YAML) $(MANIFEST_OUTPUT)
helm template install/helm/gloo $(HELMFLAGS) \
--set gateway.enabled=false,settings.integrations.knative.enabled=true | tee $@ $(OUTPUT_YAML) $(MANIFEST_OUTPUT)

install/gloo-ingress.yaml: prepare-helm
helm template install/helm/gloo $(HELMFLAGS) --values install/helm/gloo/values-ingress.yaml | tee $@ $(OUTPUT_YAML) $(MANIFEST_OUTPUT)
helm template install/helm/gloo $(HELMFLAGS) \
--set gateway.enabled=false,ingress.enabled=true| tee $@ $(OUTPUT_YAML) $(MANIFEST_OUTPUT)

.PHONY: render-yaml
render-yaml: install/gloo-gateway.yaml install/gloo-knative.yaml install/gloo-ingress.yaml
Expand Down Expand Up @@ -418,7 +422,7 @@ ASSETS_ONLY := false
# The code does the proper checking for a TAGGED_VERSION
.PHONY: upload-github-release-assets
upload-github-release-assets: build-cli render-yaml
go run ci/upload_github_release_assets.go $(ASSETS_ONLY)
GO111MODULE=on go run ci/upload_github_release_assets.go $(ASSETS_ONLY)

.PHONY: publish-docs
publish-docs: prepare-helm
Expand Down Expand Up @@ -540,14 +544,14 @@ certgen-docker-test: $(OUTPUT_DIR)/certgen-linux-amd64 $(OUTPUT_DIR)/Dockerfile.
.PHONY: build-test-chart
build-test-chart:
mkdir -p $(TEST_ASSET_DIR)
go run install/helm/gloo/generate.go $(TEST_IMAGE_TAG) $(GCR_REPO_PREFIX) "Always"
GO111MODULE=on go run install/helm/gloo/generate.go $(TEST_IMAGE_TAG) $(GCR_REPO_PREFIX) "Always"
helm package --destination $(TEST_ASSET_DIR) $(HELM_DIR)/gloo
helm repo index $(TEST_ASSET_DIR)

.PHONY: build-kind-chart
build-kind-chart:
mkdir -p $(TEST_ASSET_DIR)
go run install/helm/gloo/generate.go $(VERSION)
GO111MODULE=on go run install/helm/gloo/generate.go $(VERSION)
helm package --destination $(TEST_ASSET_DIR) $(HELM_DIR)/gloo
helm repo index $(TEST_ASSET_DIR)

Expand All @@ -565,7 +569,7 @@ build-kind-chart:
.PHONY: build-tagged-chart
build-tagged-chart:
mkdir -p $(TEST_ASSET_DIR)
go run install/helm/gloo/generate.go $(TAGGED_VERSION) $(GCR_REPO_PREFIX) "Always"
GO111MODULE=on go run install/helm/gloo/generate.go $(TAGGED_VERSION) $(GCR_REPO_PREFIX) "Always"
mkdir -p $(HELM_SYNC_DIR)/charts
helm package --destination $(HELM_SYNC_DIR)/charts $(HELM_DIR)/gloo
helm repo index $(HELM_SYNC_DIR)
Expand All @@ -584,4 +588,5 @@ fetch-tagged-helm:
#----------------------------------------------------------------------------------
.PHONY: update-licenses
update-licenses:
cd hack/utils/oss_compliance && go run main.go
# TODO(helm3): fix after we completely drop toml parsing in favor of go modules
# cd hack/utils/oss_compliance && GO111MODULE=on go run main.go
Loading

0 comments on commit fc5c370

Please sign in to comment.