[EKS] [request]: Rolling update for managed nodes when updating instance types in combination with spot instances

[nickvanwegen]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
What do you want us to build?

When you have created a node group have the ability to go to eks=>clusters=>your.cluster=>node.group=>edit and change instance types.
So it is possible to have a rolling update for managed nodes using spot configuration when changing instance types.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
What outcome are you trying to achieve, ultimately, and why is it hard/impossible to do right now? What is the impact of not having this problem solved? The more details you can provide, the better we'll be able to understand and solve the problem.

I want to be able to change the instance types of a managed node group in combination with spot.
At the moment it is possible to either define 1 instance type via a LT or multiple via node groups.
Tho when you have created the node group there is no option to change the instance types other then changing the ASG created when creating the node groups.
this in turn gives issues when using terraform for example.

Are you currently working around this issue?
For now the only option is to create a new node group migrate workload and delete the old node group.

Additional context
I came across this issue when testing managed node in combination with terraform. I think the solution provided in this request will allow tools like terraform to allow a rolling updates while also serving users without.
This issue was also partial fixed and discussed in #746 tho it only solved the issue if you use 1 instance type and define it via the LT option

Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[nickvanwegen]

Any update on this?

[mrf]

https://wangpp.medium.com/terraform-eks-nodegroups-with-custom-launch-templates-5b6a199947f It seems this might be supported if you manage your own launch group and don't let EKS create it.

[shibataka000]

@mrf As you said, we can perform rolling update to change a instance type of managed node groups when we pass instance type through the launch template. But in case of spot managed node groups, passing a instance type through the launch template is NOT recommended far as I understand.

https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html#managed-node-group-capacity-types says

When deploying your node group with the Spot capacity type that's using a custom launch template, use the API to pass multiple instance types instead of passing a single instance type through the launch template. For more information about deploying a node group using a launch template, see Launch template support.

We still cannot perform rolling update to change instances types of managed node groups which have multiple instance types.

ref: #746 (comment)

[albgus]

edit: Nevermind, this only happens if instance types is set in both the managed node group and launch template. If a launch template with a instance type is used when creating the managed node group it works (but not for lists of instance types which can be useful).

@mrf That is not true. Attempting to change the instance type in a launch template results in an error in the AWS console.

I think the only way to change the instance type of a managed node group is to create an entierly new node group with the same settings, followed by deleting the old node group.