Mechanism to allow RepoSync setup to deploy cluster scoped resource

[ethai]

We operate a large GKE cluster where we logically team each team's namespace as a tenant. We are starting to get a lot of instances where teams are deploying off the shelf vendor software and coming to us for a namespace to deploy into. Wondering if there's any prescribed strategies to allow for this type of management.

[sdowell]

My understanding here is that you have:

• A tenant, let's call team-a, with a RepoSync in namespace-a

And now team-a wants to deploy off the shelf software in namespace-b, which does not exist.

---

One suggested way to manage this would be to set up a RootSync, let's call tenant-sync, which manages tenant namespaces. The tenant-sync RootSync can sync to a repository which contains tenant configurations (namespaces, rbac, etc). A tenant can then submit a pull request to the tenant-sync git repository which sets up their new namespace, and a platform admin can review/approve the pull request.

There are two approaches to consider when setting up this sort of tenant model, mentioned in more detail here. In summary:

• The "centralized control" approach is to manage both the namespace and RepoSync(s) with the tenant-sync RootSync. This gives the platform admin full control/visibility of the setup, since any changes to the RepoSync(s)/Namespace must be approved by the platform admin.
• The "delegated control" approach is to just manage the namespace and RoleBindings with the tenant-sync RootSync. The RoleBinding should give the tenant permission to create RepoSyncs and any other resources they want to manage in their Namespace. This delegates permissions to the tenant, so there is a tradeoff of less control/visibility for the platform admin in exchange for less need to review changes.

[ethai]

specifically it's RepoSync in namespace-a wants to setup the resources ie. ClusterRole / ClusterRoleBinding that the ns-reconciler is preventing

https://github.com/oracle/coherence-operator/releases/download/v3.3.5/coherence-operator.yaml

The goal from our platform team perspective was to simply up the reposync in namespace-a then allow and configure the automatic service account with just the right amount of ClusterRole to allow for the resources it'll need. CRDs, ClusterRole, ClusterRoleBinding.

i can probably rejig and hand out a rootsync object to another github repo of their choosing, but then that'll overprivileged them with Clusteradmin role which I don't want.

[sdowell]

It is by design that RepoSyncs can only manage namespace-scoped resources and not cluster-scoped resources.

For your use case you might consider setting up a RootSync with more restricted permissions using the spec.overrides.roleRefs field

Edit: See https://github.com/GoogleContainerTools/kpt-config-sync/blob/main/docs/design-docs/02-custom-root-reconciler-clusterrole.md for more context and examples

[ethai]

this is great I'll pivot and refactor to use Rootsync resource to satisfy my use case.

We're on 1.17.1 of config management do you recall when this particular feature you mentioned was added?

[sdowell]

This feature was added with Config Sync v1.17.0