include platform flag or more granular rbac controls in helm chart for openshift

[ctml91]

Describe the solution you'd like
To add a flag for the platform in helm chart to handle appropriate RBAC and annotations or more granular RBAC . In OpenShit the default SCCs do not grant privileges to allow one to use seccomp. #842 discusses it a bit, and the recommendation now included in the docs is to apply some additional privileges in one of the roles though manually (after install?). #1127 has a similar example, so I've adjusted it and applied the below CR/CRB prior to installing via helm in order to get the installation to complete.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gatekeeper-manager-openshift-role
  namespace: gatekeeper-system
rules:
- apiGroups:
  - security.openshift.io
  resourceNames:
    - anyuid
  resources:
    - securitycontextconstraints
  verbs:
    - use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    gatekeeper.sh/system: "yes"
  name: gatekeeper-manager-openshift-rolebinding
  namespace: gatekeeper-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gatekeeper-manager-openshift-role
subjects:
- kind: ServiceAccount
  name: gatekeeper-admin
  namespace: gatekeeper-system
- kind: ServiceAccount
  name: gatekeeper-admin-upgrade-crds
  namespace: gatekeeper-system
- kind: ServiceAccount
  name: gatekeeper-update-namespace-label
  namespace: gatekeeper-system

It would be preferable to have an option to handle this in the helm chart, either automatically by setting a platform variable (e.g. openshift), or some more granular controls in the chart to specify the additional privileges required to use the OCP SCC.

In addition, whatever I try I cannot get the seccomp annotations removed during the helm install.

$ helm install -n gatekeeper-system 3.8.1 gatekeeper/gatekeeper -f values.yaml
W0614 15:15:54.166776   31622 warnings.go:70] spec.template.metadata.annotations[container.seccomp.security.alpha.kubernetes.io/manager]: deprecated since v1.19, non-functional in v1.25+; use the "seccompProfile" field instead
W0614 15:15:54.311431   31622 warnings.go:70] spec.template.metadata.annotations[container.seccomp.security.alpha.kubernetes.io/manager]: deprecated since v1.19, non-functional in v1.25+; use the "seccompProfile" field instead

My values.yaml has the default annotation removed by setting it to {}, I've even tried an arbitrary one to try and prevent the default seccomp annotation from getting added.

podAnnotations:
  {gatekeeper-platform: openshift}

And the deployment created by helm still adds this annotation, can't figure out why because the annotation is only a default setting anything else should override that - not be appended to it? Because the annotation is still included, it means it fails to come up on OCP. I inspected the helm chart but from the deployment templates can't figure out why it's still included.

  template:
    metadata:
      annotations:
        container.seccomp.security.alpha.kubernetes.io/manager: runtime/default
        gatekeeper-platform: openshift

$ oc get events
LAST SEEN   TYPE      REASON              OBJECT                                                MESSAGE
7m45s       Warning   FailedCreate        replicaset/gatekeeper-audit-fd5df4946                 Error creating: pods "gatekeeper-audit-fd5df4946-" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/manager: Forbidden: seccomp may not be set,....

Environment: OCP 4.10

• Gatekeeper version: 3.8.1
• Kubernetes version: (use kubectl version): v1.23.5

[erezo9]

@ctml91 I also experienced this issue
I recently added a PR to add rbac to each of the SA
also, to remove the annotation, just put podAnnotations: null and it should work fine

[ritazh]

I just tried to reproduce this by updating manifest_staging/charts/gatekeeper/values.yaml to

podAnnotations:
  {gatekeeper-platform: openshift}

After running helm template manifest_staging/charts/gatekeeper > tmp I see the following in gatekeeper/templates/gatekeeper-controller-manager-deployment.yaml

template:
    metadata:
      annotations:
        gatekeeper-platform: openshift

helm version
version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.6"}

If I update values.yaml to

podAnnotations: {}

The generated yaml is:

  template:
    metadata:
      annotations:
        {}

And installing with helm install manifest_staging/charts/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace, I get the following pod yaml without any annotations

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2022-06-24T01:29:21Z"
  generateName: gatekeeper-controller-manager-688f5df548-
  labels:

[erezo9]

@ritazh
It only reproduces when you install the chart from a remote location and not from a local location
using GitOps solutions like ArgoCD also reproduces the issue
To reproduce use helm install --namespace gatekeeper gatekeeper -f values.yaml gatekeeper/gatekeeper , that will make it reproduce

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.