[Feature] Support setting container runtime for EKS Windows nodes

[rawahars]

What feature/behavior/change do you want?

Currently, Windows nodes running EKS 1.21 and 1.22 (to be released shortly) can be configured to use containerd as the alternate runtime. This feature request is for adding the support for the same to eksctl.
Reference- https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-windows-ami.html#containerd-bootstrap-windows

Last year, Linux support for the same was added wherein customers can define containerRuntime in their eksctl config or pass it via cli. Reference- #4051

We can use similar customer facing flag wherein Windows users can specify the runtime to be used on the node.

Why do you want this feature?

Since the support of using alternate runtime was added on Windows nodes, it would be beneficial for the customers to use the existing flags for setting the same via eksctl.

Currently, we have suggested a workaround wherein they set an environment variable via preBootstrapCommand.

How this can be enabled under the hood for Windows?

A bootstrap script is used for setting up the required configuration on the node. The same is invoked via the user data on node startup.

In order to specify a container runtime, we pass additional flag ContainerRuntime to the script. The possible values for this flag on Windows are docker and containerd.

Reference (Bootstrap script parameters)- https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-windows-ami.html#bootstrap-script-configuration-parameters

Reference (Details about container runtime flag on Windows)- https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-windows-ami.html#containerd-bootstrap-windows

[Skarlso]

Hi!

Thank you for your request! I'll take a look at this. :)

Ah, excellent. Container runtime is a supported environment property. So I see no problem in providing it.
https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-windows-ami.html#bootstrap-script-configuration-parameters

[rawahars]

Thanks @Skarlso for such a quick turnaround!

I just took a quick look at the PR which is linked to this issue.

Just wanted to mention that on Windows, the allowed values for ContainerRuntime flag are docker and containerd.
On Windows, we do not use dockerd for specifying docker runtime.

[Skarlso]

Oh you're right. Thanks for the note. Wow, why the heck is it different from linux? 🤔

[rawahars]

Well, when docker runs on Windows node, it is started as a Windows service named docker. Similarly, containerd also starts as a containerd Windows service.

One of the reasons 😄

[Skarlso]

Aaaaahh. :DDD Okay, thanks, that at least makes SOME sense. :D

[Skarlso]

@rawahars Hi! Did you have any chance of testing this? Recently, it started to fail, and we have to revert this feature until further investigation can be done. :(

[rawahars]

Hi @Skarlso, I tested both the functionalities with Windows nodes using eksctl and it worked as intended-

• Using overrideBootstrapCommand when using custom AMIs for Windows.
• Specifying containerRuntime as containerd for EKS Windows 1.21 nodes.

I was using the pre-release version: https://github.com/weaveworks/eksctl/releases/tag/v0.93.0-rc.0

Cluster config-

---
  apiVersion: eksctl.io/v1alpha5
  kind: ClusterConfig
  
  metadata:
    name: rawahars-21-test
    region: us-west-2
    version: '1.21'
  
  nodeGroups:
    - name: linux-20h2-ng
      instanceType: t2.large
      minSize: 2
      amiFamily: AmazonLinux2
    - name: windows-ctr-ng
      instanceType: m5.2xlarge
      minSize: 2
      volumeSize: 200
      ssh: # use existing EC2 key
        publicKeyName: Harsh-key-pair
      amiFamily: WindowsServer2019FullContainer
      containerRuntime: containerd
      iam:
        attachPolicyARNs:
          - arn:aws:iam::aws:policy/AmazonSSMFullAccess                 # Required to run on instance test for nodes
          - arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess              # Required to run on instance test for nodes
          - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy           # Required EKS Node IAM policy
          - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
          - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy                # Required EKS Node IAM policy

[Skarlso]

Thank you for confirming! We will fix our test accordingly and reintroduce this feature. Cheers!