operators: make Bazel authoritative while still supporting kubebuilder

[burgerdev]

Context

It's not obvious to the casual contributor how the Constellation Node Operator module interacts with its Helm chart: some resources are duplicated, even with a slight mismatch sometimes, and changing the Kubernetes resources is in the best case toily, but in the worst case might lead to bugs (diff between make deploy vs. what ends up bundled in the CLI).

On a related note: the kubebuilder project version we're using does not seem to be supported anymore for current Kubernetes versions. A scary warning reads:

[Deprecation Notice] This version is deprecated.The `go/v3` cannot scaffold projects using kustomize versions v4x+ and cannot fully support Kubernetes 1.25+.It is recommended to upgrade your project to the latest versions available (go/v4).Please, check the migration guide to learn how to upgrade your project

This PR assumes that we want to keep compatibility with kubebuilder workflows (e.g. make deploy)! If we can drop this requirement, we would not need to migrate project versions and could remove a lot of things in the operators/constellation-node-operator directory.

This PR does not fix the discrepancies between kustomized resources in o/c/config/... and helmified resources in i/c/helm/...! (e.g. the csp value) This is fundamentally hard, because we have diffs on top of the initially generated Helm chart that we would need to backport to kustomize. I suggest to ignore this for now - the embedded code generation directives are imho much more error prone than resource patches in a directory clearly related to kubebuilder Makefiles, and we're fixing the former here. This problem would obviously go away if we were to drop support for kubebuilder dev workflows.

Proposed change(s)

• Migrate kubebuilder project to go/v4 layout (guide)
• Add Bazel autogeneration for the generate and manifests targets of the kubebuilder Makefile.
• Reorganize the Constellation Helm chart to accommodate code generation (rename files, split RBAC into resources).
• Directly generate RBAC and CRDs into the Helm chart.
  • This introduces a few cosmetic changes that should not alter behaviour of the chart.
  • This also introduces whitespace errors, but that should be fixed upstream.

The commits are suitable for individual review, which might also help understanding the logical relationships better.

Additional info

• AB#3611 (unblocks removal of the kube-rbac-proxy by making changes to resources safe)

Checklist

• Add labels (e.g., for changelog category)
• Is PR title adequate for changelog?
• Link to Milestone

[netlify]

✅ Deploy Preview for constellation-docs canceled.

Name	Link
🔨 Latest commit	cc49510
🔍 Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/65aeb07a19562f00081c6dcf

[malt3]

This problem would obviously go away if we were to drop support for kubebuilder dev workflows.

I do see your point regarding overhead for supporting the kubebuilder workflows. The added benefits are quite limited. Before we invest more costs into supporting it: What is your preferred resolution?
I would be fine with either solution. Maybe to add some more background to this: I originally added the skaffolding when I was inexperienced with writing operators. If we were to build a new operator today, we might skip this support entirely.

[burgerdev]

This problem would obviously go away if we were to drop support for kubebuilder dev workflows.

I do see your point regarding overhead for supporting the kubebuilder workflows. The added benefits are quite limited. Before we invest more costs into supporting it: What is your preferred resolution? I would be fine with either solution. Maybe to add some more background to this: I originally added the skaffolding when I was inexperienced with writing operators. If we were to build a new operator today, we might skip this support entirely.

What I think is valuable to keep are the CRD and RBAC permission workflows, which is part of the reason why I ported them to Bazel in the first place. I like the idea of stating the permissions you need right next to the location where you use them in code, and nobody should be hand-crafting any CRD yaml :) What I can definitely live without is the Makefile, the config dir etc - if nobody is using this for rapid feedback operator development, I'd say we're better off without that.

[malt3]

What I can definitely live without is the Makefile, the config dir etc - if nobody is using this for rapid feedback operator development, I'd say we're better off without that.

Sounds good to me. Let's go with that.