Added T3 Unlimited support

[efim-a-efim]

Description

Added support of T3 Unlimited flag for nodegroups. See #2368

This sets the flag:

nodeGroups:
- name: ng1:
  instancesDistribution:
    instanceTypes:
      - t3.large
      - t3a.large
  cpuCredits: "unlimited"

To disable this functionality, use cpuCredits: "standard".

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the userdocs directory)
• Manually tested
• Added labels for change area (e.g. area/nodegroup), target version (e.g. version/0.12.0) and kind (e.g. kind/improvement)
• Make sure the title of the PR is a good description that can go into the release notes

[michaelbeaumont]

otherwise nice

[joergrech]

Should this feature already work in eksctl 0.24.0-rc.1 with AWS Kubernetes? I'm getting the following error

Error: cpuCredits option set for nodegroup, but it has no t2/t3 instance types

when executing

eksctl create nodegroup -f ./kubernetes/EKS/eksctl-nodegroups.yaml

with the following content for ./kubernetes/EKS/eksctl-nodegroups.yaml:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: <CLUSTER_NAME>
  region: eu-central-1

nodeGroups:
  - name: small-spot-nodes2
    availabilityZones: ["eu-central-1b"]
    labels:
      pool:     small-spot-nodes2
      disksize: small
      nodesize: small
      nodetype: spot
    instancesDistribution:
      instanceTypes: ["t3.small","t3a.small"]
      onDemandBaseCapacity: 0
      onDemandPercentageAboveBaseCapacity: 0
      spotAllocationStrategy: "lowest-price"
      spotInstancePools: 2
    cpuCredits: "standard"
    minSize: 1
    maxSize: 20
    desiredCapacity: 1
    volumeSize: 20
    iam:
      withAddonPolicies:
        autoScaler: true

NOTE: my goal is to prevent t2/t3 spot instances to generate additional CPUCredits (the default seems to be "unlimited")
NOTE2: it does not matter if the nodegroup exists or not (tested by changing the name)
NOTE3: cpuCredits does not work for managedNodeGroups (error unmarshaling JSON: while decoding JSON: json: unknown field "cpuCredits")
NOTE4: I upgraded my Cluster from 1.16 to 1.17 today via the AWS EKS control plane

[michaelbeaumont]

The default certainly shouldn't have changed with this PR.

[michaelbeaumont]

@joergrech Can you try adding instanceType: mixed to the nodegroup?

[joergrech]

OK, that worked with nodeGroups - not sure how/if this can work with managedNodeGroups. I'll check tomorrow if this is really set in AWS.
Thanks!

[joergrech]

Seems to work - unlimited is disabled in EC2
Btw. "unlimited" is set as default from AWS side - see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-standard-mode-concepts.html
T3 and T3a instances are launched as unlimited by default. T2 instances are launched as standard by default

[michaelbeaumont]

Yeah I saw that too, thanks @joergrech!