Allow for annotations of individual resources

[chauncey-garrett]

Summary

We need the capability to individually annotate the Service resource.

Use cases

In our particular use case, we need to provide the following annotation on the Service resource:

  annotations:
    cloud.google.com/neg: '{"ingress":true}'

This annotation is automatically added by GKE. Since we use Gitops and the annotation doesn't exist, Argo CD sees the resource as being perpetually out of sync.

Another use case would be to add appropriate Workload Identity annotations to a SA.

Proposed solution

While the capability to add annotations to all resources currently exists through .Values.scim.annotations, the chart should allow annotations on each individual k8s resource (e.g., Service, Deployment, etc.).

Is there a workaround to accomplish this today?

No, not without using kustomize.

References and prior work

For example, https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/values.yaml allows annotations on individual resources.