Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(aws-eks): Support bottlerocket arm64 AMI's #14466

Closed
gfina opened this issue Apr 30, 2021 · 5 comments · Fixed by #15454
Closed

(aws-eks): Support bottlerocket arm64 AMI's #14466

gfina opened this issue Apr 30, 2021 · 5 comments · Fixed by #15454
Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service bug This issue is a bug. effort/small Small work item – less than a day of effort p2

Comments

@gfina
Copy link

gfina commented Apr 30, 2021

We currently only use x86_64 for bottlerocket AMI's even though sometimes arm64 is needed (for graviton instance types for example).


Originaly reported as

EKS: using add_auto_scaling_group_capacity fails OS BOTTLEROCKET and Graviton Instance type

description of the bug:

EKS: using add_auto_scaling_group_capacity fails with machine_image_type=eks.MachineImageType.BOTTLEROCKET and Graviton Instance type

    cluster.add_auto_scaling_group_capacity(id='BottlerocketNodes', instance_type=ec2.InstanceType('t4g.large'),
                                            machine_image_type=eks.MachineImageType.BOTTLEROCKET,
                                              min_capacity=2.0)

Reproduction Steps

setup an EKS cluster with CDK and then add to it an addition capacity, with BOTTLEROCKET and graviton instance using this:

   cluster.add_auto_scaling_group_capacity(id='BottlerocketNodes', instance_type=ec2.InstanceType('t4g.large'),
                                                machine_image_type=eks.MachineImageType.BOTTLEROCKET,
                                                  min_capacity=2.0)

cdk --version
1.101.0 (build 149f0fc)

The Autoscaling throws and error:
Launching a new EC2 instance. Status Reason: The architecture 'arm64' of the specified instance type does not match the architecture 'x86_64' of the specified AMI. Specify an instance type and an AMI that have matching architectures, and try again. You can use 'describe-instance-types' or 'describe-images' to discover the architecture of the instance type or AMI. Launching EC2 instance failed

What did you expect to happen?

I would expect an AutoScaling group to be created with 2 instances on t4g.large.

What actually happened?

Launching a new EC2 instance. Status Reason: The architecture 'arm64' of the specified instance type does not match the architecture 'x86_64' of the specified AMI. Specify an instance type and an AMI that have matching architectures, and try again. You can use 'describe-instance-types' or 'describe-images' to discover the architecture of the instance type or AMI. Launching EC2 instance failed

Environment

  • **CDK CLI Version : ** 1.101.0 (build 149f0fc)
  • Framework Version:
  • Node.js Version: v12.10.0
  • OS : Mac OS Big Sur 10.2.3
  • Language (Version): Python 3.8.7 (v3.8.7:6503f05dd5, Dec 21 2020, 12:45:15)

Other

This code swapping out the instance works as expected:

        cluster.add_auto_scaling_group_capacity(id='BottlerocketNodes', instance_type=ec2.InstanceType('t3.xlarge'),
                                                machine_image_type=eks.MachineImageType.BOTTLEROCKET,
                                                  min_capacity=2.0)

So it appears that CDK is not making the correct AMI decision with Bottlerocket and ARM instance types. However is for intel instance types.


This is 🐛 Bug Report

@gfina gfina added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 30, 2021
@github-actions github-actions bot added the @aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service label Apr 30, 2021
@gfina gfina changed the title EKS: using add_auto_scaling_group_capacity fails with machine_image_type=eks.MachineImageType.BOTTLEROCKET and Graviton Instance type EKS: using add_auto_scaling_group_capacity fails OS BOTTLEROCKET and Graviton Instance type Apr 30, 2021
@iliapolo
Copy link
Contributor

iliapolo commented May 5, 2021

Looks like we are always assuming x86_64 for bottlerocket.

this.amiParameterName = `/aws/service/bottlerocket/aws-k8s-${this.kubernetesVersion}/x86_64/latest/image_id`;

@pahud You have some insight on this? Can we use arm64 in the SSM parameter type when needed?

@iliapolo iliapolo added the investigating This issue is being investigated and/or work is in progress to resolve the issue. label May 5, 2021
@rmsy
Copy link

rmsy commented Jun 17, 2021

Just wanted to ask about the same thing, but for the ECS variant of the image:

this.amiParameterName = `/aws/service/bottlerocket/${this.variant}/x86_64/latest/image_id`;

An AMI does exist for aws-ecs-1/arm64. If this is something straightforward to submit a PR for, I would be willing to give it a shot.

@pahud
Copy link
Contributor

pahud commented Jun 18, 2021

Looks like we are always assuming x86_64 for bottlerocket.

this.amiParameterName = `/aws/service/bottlerocket/aws-k8s-${this.kubernetesVersion}/x86_64/latest/image_id`;

@pahud You have some insight on this? Can we use arm64 in the SSM parameter type when needed?

According to the bottlerocket github repo, we do have arm64 support now.
https://github.com/bottlerocket-os/bottlerocket#architectures

aws ssm get-parameters --names /aws/service/bottlerocket/aws-k8s-1.20/arm64/latest/image_id
{
    "Parameters": [
        {
            "Name": "/aws/service/bottlerocket/aws-k8s-1.20/arm64/latest/image_id",
            "Type": "String",
            "Value": "ami-0212d00ded9cb67f4",
            "Version": 2,
            "LastModifiedDate": "2021-05-20T01:14:42.129000+08:00",
            "ARN": "arn:aws:ssm:ap-northeast-1::parameter/aws/service/bottlerocket/aws-k8s-1.20/arm64/latest/image_id",
            "DataType": "text"
        }
    ],
    "InvalidParameters": []
}

I support the PR for it.

@iliapolo
Copy link
Contributor

@pahud Thanks. Converting this to a feature request. Happy to accept a contribution here.

@iliapolo iliapolo changed the title EKS: using add_auto_scaling_group_capacity fails OS BOTTLEROCKET and Graviton Instance type (aws-eks): Support bottlerocket arm64 AMI's Jun 20, 2021
@iliapolo iliapolo added effort/small Small work item – less than a day of effort p2 and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. needs-triage This issue or PR still needs to be triaged. labels Jun 20, 2021
@mergify mergify bot closed this as completed in #15454 Aug 23, 2021
mergify bot pushed a commit that referenced this issue Aug 23, 2021
Add support for Bottlerocket AMIs on ARM64 architecture, including AWS Graviton2 instances. 

Closes #14466 

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Aug 26, 2021
Add support for Bottlerocket AMIs on ARM64 architecture, including AWS Graviton2 instances. 

Closes aws#14466 

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this issue Sep 6, 2021
Add support for Bottlerocket AMIs on ARM64 architecture, including AWS Graviton2 instances. 

Closes aws#14466 

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
david-doyle-as24 pushed a commit to david-doyle-as24/aws-cdk that referenced this issue Sep 7, 2021
Add support for Bottlerocket AMIs on ARM64 architecture, including AWS Graviton2 instances. 

Closes aws#14466 

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service bug This issue is a bug. effort/small Small work item – less than a day of effort p2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants