Define how flags / cli / configuration should look for multi node

[fabriziopandini]

Multi node is going to increase complexity of the config options available in kind (see #130).

It should be defined how this will impact on flags / cli, considering at least following categories of users/use cases:

• "beginner": just give me a working, single node cluster.
• "intermediate": give me a working cluster with more than one node with some Kubernetes feature enabled
• "advanced": give me a working cluster with (almost) everything customized
• "developer": let me spin a cluster - any of the above variants - and give me full control of the actions executed after creating the cluster (see Introduce actions #131)

[fabriziopandini]

A first brainstorming:

1. "beginner": use kind create cluster as of today
2. "intermediate": use following kind create cluster flags to change the number of node
   --worker-nodes number of worker nodes
   --control-plane-nodes number of control-plane nodes
   --external-etcd to create and extenal-etcd cluster
3. "intermediate": additional flags could be added to trigger "well-known" changes to the kubeadm config file
   --kube-dns to use kube-dns instead of core-dns
   --dynamic-kubelet-config to configure
   etc.
4. "intermediate": additional flags could be added to link to one or more patches file to be applied to the kubeconfig files
   --patches
5. "advanced": use following kind create cluster flags to get full control
   --config path to a config file
6. "developer": use following kind create cluster flags
   --action to specify which action kind should perform after container creation ("none" means bare containers)

Priority among different options is a topic to be addressed too; in this case, the problem could be a little bit different than what currently under discussion for ComponentConfig because the most of the suggested flags does not map to single entries in the config file, but instead are shorcut for "complex" seetings in the config / impacting more entries at the same time

[BenTheElder]

In no particular order CCing some others for their thoughts on this: @neolit123 @liztio @munnerz

I think roughly the plan outlined above makes a lot of sense, if we go this route we should group the flags by "usage level" in the CLI help text.

[neolit123]

random thoughts:

--worker-nodes number of worker nodes
--control-plane-nodes number of control-plane nodes
--external-etcd to create and extenal-etcd cluster

while we can create replicas based on a template, there has to be a way to configure parameters for them - i.e. passing a template, or a ConfigMap (like proposed in the multi-node PR).

while passing a number of certain node types may satisfy the basic use case, soon after the user will ask "but how do i configure these replicas?"

--kube-dns to use kube-dns instead of core-dns
--dynamic-kubelet-config to configure

seems to me that these are fairly low priority, in fact i think we should only expose them when we reach the point of user demand. DKC is problematic, while kube-dns will see less use in time.

"intermediate": additional flags could be added to link to one or more patches file to be applied to the kubeconfig files
--patches

what if the user wants to patch something all the way down to the api-server manifest on all CP replicas?

"advanced": use following kind create cluster flags to get full control
--config path to a config file

i can consider config file use as pretty standard for beginners too, so my design would have been to use a config as swiss army knife for everything but only expose CLI flags that make sense for the sake of not creating a config file.

"developer": use following kind create cluster flags
--action to specify which action kind should perform after container creation ("none" means bare containers)

we need to provide a way to target a container or target a set of replicas.
i wonder if we should not expose actions from the CLI, but either accept an action file (config format) or support Actions in the same config that holds the Nodes objects.

[fabriziopandini]

According to feedbacks/latest PR merged:

1. "beginner": use kind create cluster as of today > done
2. "intermediate": expose flags to change the number of nodes > Add cli flags for multinode #247
   3,4. "intermediate": expose flags to trigger "well-known" config > not a priority, use 5 instead
3. "advanced": use following --config path flags to get full control > done
4. moving to a separate issue for better tracking

[BenTheElder]

Related: #340

We should discuss this again on Monday. @neolit123 had some interesting points about possibly moving some of the flags we already have into config.

[neolit123]

i think the ideas to support some of the flags such as --num-x-nodes on the default kind CLI is running outdated. i'm suggesting to close this ticket?

[neolit123]

/unassign

[BenTheElder]

This seems to be working well with the config files, they're very simple if you just want multiple nodes:

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: worker
- role: worker