[EKS] [request]: allow to change configuration on vpc-cni addon

[fasher]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Adding vpc-cni as a managed add-on is very nice. however right now you cannot change any configuration through it.
for example: to set AWS_VPC_K8S_CNI_EXTERNALSNAT I have to modify aws-node daemonset.
it would be nice if this can be controlled through the add-on

Which service(s) is this request for?
EKS

Are you currently working around this issue?
Yes

[mveitas]

What sort of configuration are we going to be able to set? For example, we are starting to require that all workloads running in a cluster specify resource values for cpu and memory. Is this something that will be included?

[kishoregv]

It would be nice if we can set labels and annotations

[chrissng]

Able to customize warm_ip_target and/or minimum_ip_target would be useful.

[jbilliau-rcd]

Agreed; I don't want to manage the CNI lifecycle myself, but that's my only option if i want to control WARM_IP_TARGET and MINIMUM_IP_TARGET, which the defaults take up a LOT of IP's from the VPC.

[liammurray]

Also would be useful for enabling SG for pods (which I do currently via separate k8s patch step)

[jwenz723]

I would love to see the ability to enable/configure Custom Networking using the vpc-cni addon.

Not sure if that would be handled by this issue or by #867

[sriramranganathan]

Amazon EKS team recently announced the general availability of advanced configuration feature for managed add-ons. You can now pass in advanced configuration for cluster add-ons, enabling you to customize add-on properties not handled by default settings. Configuration can be applied to add-ons either during cluster creation or at any time after the cluster is created.

Custom configuration will also allow you to set AWS_VPC_K8S_CNI_EXTERNALSNAT environment variable for the Amazon VPC CNI add-on.

To learn more about this feature, check out this blogpost - https://aws.amazon.com/blogs/containers/amazon-eks-add-ons-advanced-configuration/

Check out the Amazon EKS documentation - https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html

[miranda-delfiacco]

Thank you for adding this feature, is there any chance WARM_IP_TARGET will be a configurable option in the future or should I open a new request for that?

[mikestef9]

That's coming soon with v1.12.0-eksbuild.2

[jdn5126]

Just wanted to update here that in v1.13.0-eksbuild.1, you will also be able to configure tolerations, affinity, liveness and readiness probe timeouts, and ENIConfigs

[avnerv]

what about annotations and labels?

[jdn5126]

Pod labels and pod annotations are not currently configurable via EKS managed addons. I have made a note to add these for future versions

[jdn5126]

Configuring additional pod labels and pod annotations via EKS managed addons will ship with VPC CNI v1.15.5, for which the release is currently in progress

[avnerv]

jfyi, for v1.16.0 is working, ty!