Remove restriction of single cluster per namespace

[staebler]

It makes for a bad user experience if a user can break the functionality of an existing Cluster by adding another Cluster to the same namespace. Are there use cases around the restriction of a single cluster per namespace? I see that this restriction came out of kubernetes-retired/kube-deploy#463, but I don't see any arguments in that issue for why this restriction is needed.

Red Hat would like to allow users to be able to create multiple Clusters in a namespace so that the user can have the Clusters can all use the same Secrets around things like account credentials. The alternative is that the user would be required to copy their account credentials for every Cluster that they wish to create.

See #145 for a discussion on using labels to associate MachineSets and MachineDeployments with Clusters when a namespace has multiple Clusters.

Here is an example of a place where a single Cluster per namespace is required in the current implementation.

cluster-api/pkg/controller/machine/controller.go

Lines 153 to 167 in 5e323ca

	func (c *MachineControllerImpl) getCluster(machine *clusterv1.Machine) (*clusterv1.Cluster, error) {
	clusterList, err := c.clientSet.ClusterV1alpha1().Clusters(machine.Namespace).List(metav1.ListOptions{})
	if err != nil {
	return nil, err
	}
	switch len(clusterList.Items) {
	case 0:
	return nil, errors.New("no clusters defined")
	case 1:
	return &clusterList.Items[0], nil
	default:
	return nil, errors.New("multiple clusters defined")
	}
	}

[medinatiger]

can you elaborate the Red Hat use case in more detail? Is cluster object live outside cluster?

[staebler]

@medinatiger Red Hat will have what we are loosely terming a root cluster where users can create Cluster and MachineSets. The Red Hat product will create a remote OpenShift cluster in AWS for each Cluster in the root cluster. A Cluster in the root cluster will have in its ProviderConfig a reference to a Secret that will contain AWS account credentials to use to create the remote OpenShift cluster. It would be a better experience for Red Hat's users if they could create that Secret once in a single namespace and have all of the Clusters that they create in that namespace share that Secret. That way, if there are changes that need to be made to the account credentials, the user has only one place that they need to make the changes.

When the Red Hat product creates a remote OpenShift cluster corresponding to a Cluster in the root cluster, the Red Hat product will also create Cluster and MachineSets in the remote cluster. The Cluster and MachineSets in the root cluster will remain the ultimate source of truth, though.

[dgoodwin]

We talked about opening an issue for this on call today but this one may be sufficient. Essentially the proposal is to revisit the decision taken in kubernetes-retired/kube-deploy#463.

Reasons given were:

• Difficult for the apiserver to actually validate that only one cluster is created in a namespace unless we enforce a single cluster name.
• No need to copy shared objects. (Secrets, RBAC policies, etc)

This spawned from discussion around how to link machines to machinesets and clusters. The proposed solution resulting from that was a required local reference from machineset/machinedeployment/machine to an owning cluster, enforced by the apiserver validation. No use cases were uncovered on call where machines are in use but a cluster does not exist for them to belong to.

If however we maintain the assumption that only one cluster can exist per namespace, this link remains unnecessary and this additional step of setting a local object cluster reference on your machine types would not be required.

CC @craigtracey @roberthbailey @krousey

[wangzhen127]

/cc @wangzhen127

[roberthbailey]

De-duping with #41.

/close

[k8s-ci-robot]

@roberthbailey: Closing this issue.

In response to this:

De-duping with #41.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.