Pinned version of ingress-nginx fails to starts with latest version of Kubernetes

[jessebrennan]

Running

./terra-app-local.sh install -f apps/jupyter/app.yaml

failed for me until I downgraded the version of Kubernetes I was using.

I'm a Kube noob so it took me some time to figure this out because the error is pretty opaque. Taking the helm upgrade command from terra-app-local.sh and running it with debug flags gives:

helm upgrade --install nginx ingress-nginx/ingress-nginx -n nginx --version 3.23.0 --set 'controller.service.externalIPs[0]=192.168.49.2' --debug
history.go:56: [debug] getting history for release nginx
upgrade.go:142: [debug] preparing upgrade for nginx
upgrade.go:150: [debug] performing update for nginx
upgrade.go:322: [debug] creating upgraded release for nginx
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ServiceAccount
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRole
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRoleBinding
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" Role
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" RoleBinding
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission-create" Job
client.go:328: [debug] jobs.batch "nginx-ingress-nginx-admission-create" not found
client.go:128: [debug] creating 1 resource(s)
client.go:529: [debug] Watching for changes to Job nginx-ingress-nginx-admission-create with timeout of 5m0s
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-create: ADDED
client.go:596: [debug] nginx-ingress-nginx-admission-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-create: MODIFIED
client.go:596: [debug] nginx-ingress-nginx-admission-create: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-create: MODIFIED
client.go:596: [debug] nginx-ingress-nginx-admission-create: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-create: MODIFIED
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ServiceAccount
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRole
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRoleBinding
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" Role
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" RoleBinding
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission-create" Job
client.go:218: [debug] checking 10 resources for changes
client.go:501: [debug] Looks like there are no changes for ServiceAccount "nginx-ingress-nginx"
client.go:510: [debug] Patch ConfigMap "nginx-ingress-nginx-controller" in namespace nginx
client.go:501: [debug] Looks like there are no changes for ClusterRole "nginx-ingress-nginx"
client.go:501: [debug] Looks like there are no changes for ClusterRoleBinding "nginx-ingress-nginx"
client.go:501: [debug] Looks like there are no changes for Role "nginx-ingress-nginx"
client.go:501: [debug] Looks like there are no changes for RoleBinding "nginx-ingress-nginx"
client.go:501: [debug] Looks like there are no changes for Service "nginx-ingress-nginx-controller-admission"
client.go:501: [debug] Looks like there are no changes for Service "nginx-ingress-nginx-controller"
client.go:510: [debug] Patch Deployment "nginx-ingress-nginx-controller" in namespace nginx
client.go:510: [debug] Patch ValidatingWebhookConfiguration "nginx-ingress-nginx-admission" in namespace 
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ServiceAccount
client.go:328: [debug] serviceaccounts "nginx-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRole
client.go:328: [debug] clusterroles.rbac.authorization.k8s.io "nginx-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" ClusterRoleBinding
client.go:328: [debug] clusterrolebindings.rbac.authorization.k8s.io "nginx-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" Role
client.go:328: [debug] roles.rbac.authorization.k8s.io "nginx-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission" RoleBinding
client.go:328: [debug] rolebindings.rbac.authorization.k8s.io "nginx-ingress-nginx-admission" not found
client.go:128: [debug] creating 1 resource(s)
client.go:299: [debug] Starting delete for "nginx-ingress-nginx-admission-patch" Job
client.go:128: [debug] creating 1 resource(s)
client.go:529: [debug] Watching for changes to Job nginx-ingress-nginx-admission-patch with timeout of 5m0s
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-patch: ADDED
client.go:596: [debug] nginx-ingress-nginx-admission-patch: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:557: [debug] Add/Modify event for nginx-ingress-nginx-admission-patch: MODIFIED
client.go:596: [debug] nginx-ingress-nginx-admission-patch: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
upgrade.go:431: [debug] warning: Upgrade "nginx" failed: post-upgrade hooks failed: timed out waiting for the condition
Error: UPGRADE FAILED: post-upgrade hooks failed: timed out waiting for the condition
helm.go:84: [debug] post-upgrade hooks failed: timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
	helm.sh/helm/v3/cmd/helm/upgrade.go:199
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.3.0/command.go:902
main.main
	helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	runtime/proc.go:255
runtime.goexit
	runtime/asm_amd64.s:1581

I ended up trying an older version of Kubernetes when I started minikube, and this worked:

minikube start --mount --mount-string="data:/data" --kubernetes-version=v1.19.16