Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create multiple empty node groups by default #84

Closed
jrsdav opened this issue Jul 25, 2022 · 2 comments
Closed

Create multiple empty node groups by default #84

jrsdav opened this issue Jul 25, 2022 · 2 comments

Comments

@jrsdav
Copy link
Contributor

jrsdav commented Jul 25, 2022

The module should be refactored to create multiple node groups using set(s) of AWS EC2 Instance Classes that correlate to a StreamNative cluster size.

The node groups should have a desired_capacity setting of 0, allowing them to scale up/down according to what workloads are being scheduled by the StreamNative control plane.

Depending on regional availability:

  • c6i.large, c6i.xlarge, c6i.2xlarge, c6i.4xlarge

Or:

  • c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge

Other requirements:

  • The name structure should match the naming pattern "snc-*-pool*"
    • Have some variation of the instance class in their name
    • The node group name should have a unique suffix

Nice to have:

  • Include a taint that designates its instance class (micro, tiny, small, medium, large, xlarge, xxlarge). This will require some coordination with the control plane, so it should not be used until it is supported on workloads. Example value structure:
taints = [{
      key    = "reserveGroup"
      value  = "micro"
      effect = "NO_SCHEDULE"
    }]
@gengmao gengmao mentioned this issue Aug 3, 2022
2 tasks
@gengmao
Copy link
Contributor

gengmao commented Aug 3, 2022

@jrsdav Inspired by this comment, I found the terraform-aws-eks module actually supports creating launch template with instance types and don't specify instance types when creating node group via a set_instance_types_on_lt variable which is available in v17.4.0. That will allow us to update instance types in the launch template via Terraform without impacting existing node groups, then trigger an Instance Refresh manually to rotate instances.

The drawback of setting set_instance_types_on_lt is it breaks backward compatibility on existing clusters/node groups. Hence I created #85 to simplify updating instance types for existing clusters.

So different strategies for different clusters.

@gengmao
Copy link
Contributor

gengmao commented Feb 15, 2023

This was done with #91

@gengmao gengmao closed this as completed Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants