Namespace pod tolerations

[amisevsk]

What does this PR do?

Adds support for namespace-level configuration of pod tolerations and node selector via namespace annotations

• "controller.devfile.io/pod-tolerations"
• "controller.devfile.io/node-selector"

The value of the tolerations annotation must be json-formatted []corev1.Toleration. The value of the node-selector annotation must be map[string]string. If these annotations are applied to a namespace, their values are used for all pods created for workspaces in that namespace. This includes:

• The workspace pod itself
• The pod created as part of the PVC cleanup job
• The async storage server deployment

What issues does this PR fix or reference?

Closes #614 (assuming this implementation is sufficient)

Is it tested? How?

Instructions written for minikube, on other clusters node name should be changed

1. Taint a node and add a label to it

   # Apply a taint to the minikube node
   kubectl taint nodes minikube key1=value1:NoSchedule
   # Apply a new label to the minikube node
   kubectl label nodes minikube test-node-selector=test

2. Add annotations to the namespace to be used:

   metadata:
     annotations:
       controller.devfile.io/node-selector: |
         {
           "test-node-selector": "test"
         }
       controller.devfile.io/pod-tolerations: |
         [
           {
             "key": "key1",
             "operator": "Equal",
             "value": "value1",
             "effect": "NoSchedule"
           }
         ]

   Note: formatting can be finicky, if annotations cannot be unmarshalled, all DWO activity in the namespace fails.
3. Create DevWorkspace and verify it can start in the cluster (and deployment has appropriate nodeSelector/tolerations)

   kubectl apply -f samples/theia-next.yaml

4. Once workspace is started, delete it and verify cleanup succeeds

   kubectl delete dw theia-next --wait
   

PR Checklist

• E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
  • v8-devworkspace-operator-e2e: DevWorkspace e2e test
  • v8-che-happy-path: Happy path for verification integration with Che

[JPinkney]

I can't really comment on the approach since I'm not familiar with pod tolerations/node selectors but I tried it out and everything is working and from a code perspective it looks fine

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amisevsk, JPinkney

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JPinkney,amisevsk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[amisevsk]

/test v8-devworkspace-operator-e2e