feat(lambda-layer-kubectl): custom lambda layer for kubectl based on EKS version of the cluster

[robertd]

This is an early WIP/POC

---

All Submissions:

• Have you followed the guidelines in our Contributing guide?

Adding new Unconventional Dependencies:

• This PR adds new unconventional dependencies following the process described here

New Features

• Have you added the new feature to an integration test?
  • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[gitpod-io]

[pahud]

Awesome!

[pahud]

Suggested change

	KUBECTL_VERSION: props.kubectlVersion,
	HELM_VERSION: '3.8.2',
	KUBECTL_VERSION: props?.kubectlVersion ?? '1.20.0',
	HELM_VERSION: props?.helmVersion ?? '3.8.1',

If props undefined, the default versions should be the original versions, wdyt?

[robertd]

Will KubectlLayer ever be used outside of the Cluster constructor?

[pahud]

Good job! Thanks for the PR @robertd !

It's pretty much the same with my experimental code today. Some suggested changes provided though and some considerations as below:

1. To avoid breaking changes, maybe we should allow the props to be optional with predefined defaults of kubectl and helm CLIs. However, users are allowed to customize them if necessary.

2. We probably need a static mapping table in aws-eks for cluster version with kubectl version, e.g.
   cluster 1.22 should map to kubectl 1.22.6 according to this document so when we provision a 1.22 cluster, the kubectl of version 1.22.6 lambda layer will be provisioned.

Thoughts?

Pull request has been modified.

[pahud]

Have discussed with @neilkuan today about this issue. What we can explore next this to pass the cluster version to here so the KubectlProvider might be able to create correct layer.

aws-cdk/packages/@aws-cdk/aws-eks/lib/kubectl-provider.ts

Line 155 in 9c93ba7

handler.addLayers(new KubectlLayer(this, 'KubectlLayer'));

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 9e8529b
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[robertd]

@pahud ... few notes:

• This PR introduces direct dependency on Docker for building kubectl layer image "on the fly". In the past few months, Docker for Desktop license changed for companies and lots of folks are migrating to alternatives.

• We've observed increase in the duration of tests. For example aws-eks tests are now around ~5min (up from 30-60s). "Downstream modules" that have aws-eks and lambda-layer-kubectl as dependencies will see increase in duration too.

• Tests are still failing. Still investigating.

• We are introducing manual management of EKS version mapping in packages/@aws-cdk/aws-eks/lib/kubectl-provider.ts. I really wish kubectl download urls supported major.minor in the url and provide the latest version of the kubectl - rather than having to specify the major.minor.patch version in the url. This would make a world of difference for us (downstream users) and we wouldn't have to do manual version mapping in packages/@aws-cdk/aws-eks/lib/kubectl-provider.ts.

  • https://storage.googleapis.com/kubernetes-release/release/v1.22/bin/linux/amd64/kubectl (preferred)
    vs
    https://storage.googleapis.com/kubernetes-release/release/v1.22.9/bin/linux/amd64/kubectl (actual)

• We bumped helm to 3.9.x. I'm not expecting any issues going to 3.9.x.

[pahud]

According to the approach proposed in RFC 39, we are planning to publish @aws-cdk/lambda-layer-kubectl-X separately and this will avoid breaking changes in the aws-eks API. For users to deploy eks 1.22 clusters, @aws-cdk/lambda-layer-kubectl-122 would be required. I think this might be a better approach. wdyt?

[robertd]

@pahud Agreed. I think this PR draft was a valiant effort on our part, but I think breaking this into external module(s) is a better idea. Closing this PR draft.