Namespaces should not be synced in all clusters

[fredgate]

What would you like to be added:
The cluster controller of karmada should not automatically create a new namespace in all the clusters. It should process like others resources.

Why is this needed:
Sometimes we want that some namspaces be created only in some clusters.
The namespaces should be created only if there is a ClusterPropagationPolicy referencing it, and the target clusters should be determined from it.

[XiShanYongYe-Chang]

Hi @fredgate, thanks for your feedback!

Namespace automatically sync is controlled by namespace controller, we can disable it through --controllers flag, refer to here.

After disabling the namespace controller, we can propagate namespace resources like others.

[RainbowMango]

Yes, just like @XiShanYongYe-Chang mentioned above. The Karmada's namesspace controller is a helper controller which automatically syncs namespaces from the Karmada control plane to all registered clusters.

By the way, system-reserved namespaces won't be synced.

The cluster controller of karmada should not automatically create a new namespace in all the clusters.

Is there any particular reason that this behavior is unacceptable in your case?
Looking forward to your feedback. If this default behavior will do something bad, we can consider disabling it by default.

[fredgate]

Thanks @RainbowMango for your response.

Yes there is a reason why this behavior is annoying to me. Here is my case, but I think that it can be quite common.
I use Karmada with Flux toolkit to do GitOps and synchronize resources (mainly namespaces and HelmReleases) accross multiple clusters. Let's say that I have one cluster by customer. I have some namespaces that allow to deploy core services common to all customers. But I have also some namespaces that are specific to one customer (for example having the name of one site of the customer), so I would like that these namespaces be deployed only in a well-defined cluster.

[RainbowMango]

But I have also some namespaces that are specific to one customer (for example having the name of one site of the customer), so I would like that these namespaces be deployed only in a well-defined cluster.

@fredgate I got you! That makes sense to me. The default behavior shouldn't be annoying.

I added an agenda to the next community meeting. If no objections we can do it in the next release(should be included in v1.4.0). But before that, I suggest you can configure the controllers to disable it explicitly. Just a reminder, we are speaking Chinese in this meeting, I guess you speaking English, so if you'd like to get involved, you are welcome to propose a meeting time to this issue, and we are going to start the English meeting soon(probably this Month).

I use Karmada with Flux toolkit to do GitOps and synchronize resources (mainly namespaces and HelmReleases) accross multiple clusters.

Good to know that, and glad that Karmada helps. Do you mean you are using Karmada in production?

[fredgate]

I added an agenda to the next community meeting. If no objections we can do it in the next release(should be included in v1.4.0).

It would be perfect.

But before that, I suggest you can configure the controllers to disable it explicitly.

It seems to me that if i disable the namespace controller in karmada manager, it has no effect other than not propagate namespaces in all clusters. So it can be a work around.
However I use the Helm chart to install Karmada, and it has no option to choose the controllers of the karmada manager. If you want I can open an issue for that and provide a pull request.

I use Karmada with Flux toolkit to do GitOps and synchronize resources (mainly namespaces and HelmReleases) accross multiple clusters.

Good to know that, and glad that Karmada helps. Do you mean you are using Karmada in production?

We use Flux to manage each of our production clusters with GitOps.
I would like to centralize the management of all our clusters. So I make a POC with Karmada for that, and I'm starting to come up with something satisfying.

[RainbowMango]

However I use the Helm chart to install Karmada, and it has no option to choose the controllers of the karmada manager. If you want I can open an issue for that and provide a pull request.

Sure, please send a PR for it, we appreciate that.

It seems to me that if i disable the namespace controller in karmada manager, it has no effect other than not propagate namespaces in all clusters. So it can be a work around.

I didn't understand, I think after you disable the namespace controller, you need to propagate the namespace by creating the ClusterPropagationPolicy.

[fredgate]

I will test a change in the Karmada helm chart, and then I will submit a PR. I will add a value to choose controllers. Could you tell me what are the default controllers executed by the karmada manager.

I will test to disable the namespace controller and then to add a ClusterPropagationPolicy for each namespace that I want to synchronyze in one or more clusters. Does this controller have other roles than synchronizing namespaces ? In which case by disabling it I would lose these features.

[RainbowMango]

Could you tell me what are the default controllers executed by the karmada manager.

You can get the whole controller list from:

karmada/cmd/controller-manager/app/controllermanager.go

Lines 173 to 186 in 62a4b43

	controllers["cluster"] = startClusterController
	controllers["clusterStatus"] = startClusterStatusController
	controllers["hpa"] = startHpaController
	controllers["binding"] = startBindingController
	controllers["execution"] = startExecutionController
	controllers["workStatus"] = startWorkStatusController
	controllers["namespace"] = startNamespaceController
	controllers["serviceExport"] = startServiceExportController
	controllers["endpointSlice"] = startEndpointSliceController
	controllers["serviceImport"] = startServiceImportController
	controllers["unifiedAuth"] = startUnifiedAuthController
	controllers["federatedResourceQuotaSync"] = startFederatedResourceQuotaSyncController
	controllers["federatedResourceQuotaStatus"] = startFederatedResourceQuotaStatusController
	controllers["gracefulEviction"] = startGracefulEvictionController

These controllers (except hpa) are enabled by default(--controllers=<empty>).
If people want to disable one of them, then he/she can set the flag like --controllers=-<controller-name>.
If people want to disable more than one controllers, then he/she can set the flag like --controllers=-<one controller>,-<another controller>.

  --controllers strings                                                                                                                                                                                                      

            A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'. 
            All controllers: binding, cluster, clusterStatus, endpointSlice, execution, federatedResourceQuotaStatus, federatedResourceQuotaSync, gracefulEviction, hpa, namespace, serviceExport, serviceImport, unifiedAuth,
            workStatus.
            Disabled-by-default controllers: hpa (default [*])

[RainbowMango]

Does this controller have other roles than synchronizing namespaces ? In which case by disabling it I would lose these features.

No worries, this controller is very simple, with no coupling features with other controllers.

[fredgate]

I just created a PR to configure the controllers with the Helm chart.

But I wonder about the usefulness of the namepsace controller. Why unlike other resources, namespaces are synchronized automatically and not when referenced in a ClusterPropagationPolicy ?

[RainbowMango]

Thanks a lot, we are woking on the PR.

Why unlike other resources, namespaces are synchronized automatically and not when referenced in a ClusterPropagationPolicy ?

Good question. The namespace controller is more like a helper controller, it allows you don't need to care if a namespace is created or not when propagating workloads, especially when you need to propagate plenty of workloads to the same namespace, with this controller, you don't need to write the rule for the namespace again and again in ClusterPropagationPolicyes.

[RainbowMango]

Hi @fredgate
We just introduced a label(#2696) that can be used to label the namespace resources that should be skipped from auto-propagation.

karmada/pkg/apis/policy/v1alpha1/well_known_constants.go

Lines 13 to 21 in 026781f

	// NamespaceSkipAutoPropagationLabel is added to namespace objects to indicate if
	// the namespace should be skipped from propagating by the namespace controller.
	// For example, a namespace with the following label will be skipped:
	// labels:
	// namespace.karmada.io/skip-auto-propagation: "true"
	//
	// NOTE: If create a ns without this label, then patch it with this label, the ns will not be
	// synced to new member clusters, but old member clusters still have it.
	NamespaceSkipAutoPropagationLabel = "namespace.karmada.io/skip-auto-propagation"

For example:

apiVersion: v1
kind: Namespace
metadata:
  name: horen
  labels:
    namespace.karmada.io/skip-auto-propagation: "true"

[RainbowMango]

@fredgate
I'm going to close it as the solution has been released in v1.4.0
Yes, we are looking forward to your feedback.

[RainbowMango]

/close

[karmada-bot]

@RainbowMango: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.