Refuse to deploy global resources

[KnVerey]

Proposal

This project's mission is to make it easy to ship changes to a Kubernetes namespace and understand the result. Our contributors' guide explicitly explains that we do not support global resources:

Global resources

This project is intended to manage a namespace (or a label-delimited subsection of one). It does not officially support non-namespaced resources. In practice, we do model custom resource definitions specifically, because it helps us better handle custom resources (which typically are namespaced). However, our intent is to keep this the only exception to the rule.

However, in practice, we blindly kubectl apply any resource we don't recognize, including those that are global. That means that invoking us can have (intended or otherwise) side-effects in the global namespace. Because of this (and likely the fact that our user-facing documentation makes no mention of globals one way or the other), we've received more than one request to predeploy global resources.

I strongly believe that our deploy command should keep its mission of managing a namespace, and I would like to introduce safeguards against global side-effects as part of the transition to Krane.

Resources affected

kubectl api-resources --namespaced=false

componentstatuses
namespaces
nodes
persistentvolumes
mutatingwebhookconfigurations.admissionregistration.k8s.io
validatingwebhookconfigurations.admissionregistration.k8s.io
customresourcedefinitions.apiextensions.k8s.io
apiservices.apiregistration.k8s.io
tokenreviews.authentication.k8s.io
selfsubjectaccessreviews.authorization.k8s.io
selfsubjectrulesreviews.authorization.k8s.io
subjectaccessreviews.authorization.k8s.io
certificatesigningrequests.certificates.k8s.io
podsecuritypolicies.extensions
runtimeclasses.node.k8s.io
podsecuritypolicies.policy
clusterrolebindings.rbac.authorization.k8s.io
clusterroles.rbac.authorization.k8s.io
priorityclasses.scheduling.k8s.io
csidrivers.storage.k8s.io
csinodes.storage.k8s.io
storageclasses.storage.k8s.io
volumeattachments.storage.k8s.io

Options

A) Somehow (TBD) identify global resources during validation, and fail fast if the template set includes any. In the error, explain global resource management best practices and suggest using kubectl apply directly for them.
B) Same as the above, but build a krane command we can point them to. Given that the top-level objects are largely configuration data, this command would likely do little beyond kubectl apply anyway. There's at least one exception though: CRDs, which we already model, have meaningful status conditions. We could go through the list above to get a better idea of the potential value here. If we build such a thing, and it supports pruning, it should probably enforce label selector usage (pruning global resources unconditionally... eeek).

@Shopify/cloudx

[dturn]

We probably can do better for ClusterRoleBindings by ensuring the service account and ClusterRoles exist.

In a perfect world it would be nice if a user could point our tools at a directory like https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/deploy w/o having to manually carve it up. Though I doubt that's fully achievable without some type of --ignore-file flag or worse.

I also feel strongly that we shouldn't cut off the ability to use global objects until we support a command that handles them.(Even if it starts as a thing wrapper over kubectl apply)

[dturn]

#567 causes krane deploy to refuse to deploy global resources. I'm working on krane global-deploy command in https://github.com/Shopify/kubernetes-deploy/pull/574/files. kubernetes-deploy will maintain backwards compatibility until we delete it in v1.0.0

[dturn]

Krane deploy will fail validation if a global resource is present. #588 is tracking the creation of krane global-deploy`.