Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aws): Install AWS CLI v2, upgrade aws-iam-authenticator, remove s3cmd #6156

Merged
merged 3 commits into from
Apr 29, 2024

Conversation

jervi
Copy link
Contributor

@jervi jervi commented Feb 22, 2024

AWS CLI v1 is becoming harder to use with newer Kubernetes versions. If you use aws eks get-token for EKS authentication instead of aws-iam-authenticator, you are stuck with apiVersion: client.authentication.k8s.io/v1alpha1, because v1beta1 isn't supported by AWS CLI v1. Upgrading to v2 fixes this.

I also removed the s3cmd tool. I couldn't find a single place outside of the Dockerfiles that referenced this.

@jervi jervi changed the title feat(aws): Install AWS CLI v2 and upgrade aws-iam-authenticator feat(aws): Install AWS CLI v2, upgrade aws-iam-authenticator, remove s3cmd Feb 22, 2024
@dbyron-sf
Copy link
Contributor

LGTM. @jasonmcintosh you good with this?

@jasonmcintosh
Copy link
Member

Yes though I'd prefer to NOT have it in 1.34 as it'd break kubeconfig with out a good migration path currently documented. The versions currently included SHOULD work with v1beta (aws cli and aws-iam-authenticator) allowing a migration path to v1beta1. Note I believe v1beta1 is also deprecated at this point in time and slated for removal. aws-iam-authenticator in 0.6 removed v1alpha entirely but in older versions supported both exec APIs and aws cli 1.22 SHOULD work with both APIs as well. I'd get 1.34 out first with a docs PR saying "Caution! You need to upgrade your kubeconfig files" then in 1.35 upgrade these to remove the v1alpha1 support.

Kube exec API calls pass an env variables exec's can use to determine which API spec to support.
kubernetes-sigs/aws-iam-authenticator#439
This PR and
https://github.com/aws/aws-cli/pull/6476/files
for more information.

Docs PR for reference (which could use some language tweaks) calling out that this will break users:
spinnaker/spinnaker.io#371

@@ -26,6 +28,18 @@ install_kubectl() {
fi
}

install_awscli2() {
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWS_CLI_VERSION}.zip" -o "awscliv2.zip"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note arch here is wrong - needs to support arch argument to work on ARM instances.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The postinstall script doesn’t support arm, I just continued down that path

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm, that's... potentially problematic. I need to look at this more then...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can fix it if you want. But I don't think anyone has ever installed the package on an ARM instance. We are probably one of the last companies to actually use these packages, and now we're also migrating to k8s. Looking at the script it is also pretty lacking, without support for anything but AWS. But just let me know if you want me to add support for ARM before we merge this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wince I'd love ARM support, but you're probably right on lack of usage. That said I've missed stuff like this before and it's bitten me :(

./aws/install
rm -rf ./awscliv2.zip ./aws

curl "https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v${AWS_AIM_AUTHENTICATOR_VERSION}/aws-iam-authenticator_${AWS_AIM_AUTHENTICATOR_VERSION}_linux_amd64" -O aws-iam-authenticator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arch missing here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Member

@jasonmcintosh jasonmcintosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arch missing on the one CLI installer

@jasonmcintosh
Copy link
Member

Note looks like v1beta1 is still supported/not deprecated if I've got the right spot https://kubernetes.io/docs/reference/config-api/client-authentication.v1beta1/ though there's a plain v1. I'd have to check on state of v1 support on the cli tools.

@jervi
Copy link
Contributor Author

jervi commented Feb 27, 2024

The versions currently included SHOULD work with v1beta (aws cli and aws-iam-authenticator) allowing a migration path to v1beta1.

I'm not entirely sure why it fails for me, but running Clouddriver nightly builds (with AWS CLI v1), it fails when using kubectl v1.28 and v1beta1. It may work with older kubectl versions though. The same combination works with AWS CLI v2 (running a version with this PR applied). I'm pretty sure it should work with v1 as well given that you're running a new enough Kubernetes version.

The error I'm getting is:

no kind "ExecCredential" is registered for version "client.authentication.k8s.io/v1alpha1" in scheme "pkg/client/auth/exec/exec.go:62

Not sure where it gets v1alpha1 from, because the kubeconfig files are using v1beta1.

That being said, I'm not arguing for this to be in v1.34. I'm fine with it going in 1.35.

@jasonmcintosh
Copy link
Member

Hrmm, I'm thinking I may have missed this - 1.22 is older - Harness (aka armory) has a different build that's running 1.29. Let me do some testing, but it's possible for 1.34 we should bump to 1.29 then note that "alpha1" is deprecated, then 1.35 release with this.

@jasonmcintosh
Copy link
Member

Pointing out here:

  • AWS CLI 1.24 supports both v1alpha1 & v1beta1 - so look to upgrade that for 1.34 then this PR for 1.35. Hope to come back look at the post install scripts for support for ARM stuff.

# Conflicts:
#	Dockerfile.java11.slim
#	Dockerfile.java11.ubuntu
#	Dockerfile.slim
#	Dockerfile.ubuntu
@jasonmcintosh jasonmcintosh added the ready to merge Approved and ready for a merge label Apr 29, 2024
@mergify mergify bot added the auto merged Merged automatically by a bot label Apr 29, 2024
@jasonmcintosh jasonmcintosh merged commit 291a872 into spinnaker:master Apr 29, 2024
21 of 23 checks passed
@chris-h-phillips
Copy link
Contributor

I just tried to use the latest 1.35.2 docker image. I use the aws eks get-token method. Stages fail with: /usr/local/bin/aws: No such file or directory

Indeed if you go looking in the container:

bash: /usr/local/bin/aws: No such file or directory
bash-5.1$ ls -al /usr/local/bin/aws
lrwxrwxrwx    1 root     root            37 Aug 20 17:48 /usr/local/bin/aws -> /usr/local/aws-cli/v2/current/bin/aws
bash-5.1$ ls -al /usr/local/aws-cli/v2/current/bin
total 0
drwxr-xr-x    2 root     root            38 Aug 20 17:48 .
drwxr-xr-x    4 root     root            44 Aug 20 17:48 ..
lrwxrwxrwx    1 root     root            11 Aug 20 17:48 aws -> ../dist/aws
lrwxrwxrwx    1 root     root            21 Aug 20 17:48 aws_completer -> ../dist/aws_completer
bash-5.1$ ls -al ../dist/aws
ls: ../dist/aws: No such file or directory

@jervi I'm wondering if you might have any insight here?

@jervi
Copy link
Contributor Author

jervi commented Aug 23, 2024

@jervi I'm wondering if you might have any insight here?

You're right. Seems like an Alpine Linux problem. An easy workaround is to use spinnaker-1.35.2-ubuntu for Clouddriver instead.

$ kubectl run -i --namespace default --tty --rm debug --image=us-docker.pkg.dev/spinnaker-community/docker/clouddriver:spinnaker-1.35.2-ubuntu --restart=Never -- bash
If you don't see a command prompt, try pressing enter.
spinnaker@debug:/$ aws --version
aws-cli/2.15.22 Python/3.11.6 Linux/5.10.219-208.866.amzn2.x86_64 exe/x86_64.ubuntu.22 prompt/off

@jervi
Copy link
Contributor Author

jervi commented Aug 23, 2024

Seems like there are pre-built binaries available for Alpine now: https://pkgs.org/download/aws-cli
The docker image should probably be changed to just install AWS CLI v2 using apk.

@jervi jervi deleted the upgrade_aws_cli branch August 23, 2024 12:51
@dbyron-sf
Copy link
Contributor

Seems like there are pre-built binaries available for Alpine now: https://pkgs.org/download/aws-cli The docker image should probably be changed to just install AWS CLI v2 using apk.

Mind taking a crack at that @jervi ?

@jervi
Copy link
Contributor Author

jervi commented Aug 26, 2024

#6279

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto merged Merged automatically by a bot ready to merge Approved and ready for a merge target-release/1.35
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants