Upgrade failures on Bottlerocket 1.21.0

[gthao313]

We have identified #4135 with the Bottlerocket release 1.21.0 which could cause some Bottlerocket nodes to experience boot failures. We are working on fixing this issue and in the meantime, we have rolled back to the previous Bottlerocket release 1.20.5. We apologize for any inconvenience this may cause and thank you for your patience as we work to resolve the issue. We will provide an update on the issue as soon as possible.

[darrenwhighamfd]

Can you confirm if 1.21.0 AMIs are still in ECR as when we look to update to the latest image we see they are still found and used, as a result we are seeing the boot errors on new nodes.

[yeazelm]

Hello @darrenwhighamfd, The AMI IDs are still available. We recommend you use our SSM Parameters to find the latest image: https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id-bottlerocket.html. Currently latest will point to 1.20.5 AMIs.

We are working through the issues we found that caused boot failures and will open a new tracking issue once we believe we have resolved all the issues to provide a new image without these issues.