Helm delete doesn't support tolerations/affinities

[bburghaus]

What happened:
Helm delete will fail due to the job created by hook script pre_delete_hook.yaml if there are tolerations/affinities.

What you expected to happen:
Successful release deletion when running helm delete.

How to reproduce it (as minimally and precisely as possible):
Add tolerations/affinities to cluster, run helm delete.

Environment:

• Agones version: 1.4.0
• Kubernetes version (use kubectl version): 1.14
• Cloud provider or hardware configuration:
• Install method (yaml/helm): helm
• Troubleshooting guide log(s):
• Others:

[aLekSer]

@bburghaus Can you please provide details on what kind of tolerations do you use? Did you put them on GameServers?
There is a script which actually waits for all pods used by GameServers to be deleted install/helm/agones/scripts/delete_agones_resources.sh so it seems that this final step got stuck.
There was an issue with helm delete --purge agones time out and then helm install .. gives:

Release "agones" does not exist. Installing it now.
Error: release agones failed: object is being deleted: customresourcedefinitions.apiextensions.k8s.io "gameservers.agones.dev" already exists                                                            
make: *** [install] Error 1

In that case patching helps:

 $ kubectl patch crd/gameservers.agones.dev  -p '{"metadata":{"finalizers":[]}}' --type=merge

[bburghaus]

Hey @aLekSer, the issue is that all our nodes have taints so the job pod itself cannot be scheduled and the delete hits a timeout.

We could use either --no-hooks as a solution or agones.crds.cleanupOnDelete=false, though we want to be able to have the cleanup. We did also remove the finalizers in order to make it work.

The job should take them as separate parameters, or re-use controller tolerations/affinities.