Enable prefixes delegation by default

[jackivanov]

Prefix delegations is enabled since v1.9.0, but not by default.

What would you like to be added:
ENABLE_PREFIX_DELEGATION=true by default if > 1.9.0 add-on is installed

Why is this needed:
We want to enable this feature to deploy more pods per node, but at the moment it's not possible to automate the proccess on EKS

• EKS pre-installs 1.7.0 add-on by default for Kubernetes 1.21
• 1.9.0 upgrade can be easily handled by terraform aws_eks_addon
• ENABLE_PREFIX_DELEGATION=true, and other options (eg: WARM_PREFIX_TARGET) can't be added to EKS by proper automation (Terraform, ArgoCD) because it conflicts with the existing resources (Handle helm upgrade of AWS VPC CNI more gracefully eks-charts#57)

As a solution, the add-on configuration (env variables) can be modified through API
Also a patch resource from terraform (hashicorp/terraform-provider-kubernetes/issues/723)

[jayanthvn]

Hi @jackivanov

Prefix delegation is a very new feature from EC2 and VPC CNI so we would generally wait for production mileage before enabling it by default on all new clusters. Hence we have added the environment variable to enable it on need basis but we have plans in future to have it enabled by default.

Thanks.

[jackivanov]

@jayanthvn Thanks for the info! Can you recommend a way how we can automatically customize the add-on during EKS cluster creation? Like said in the description, automated customization is painful, and there's always a conflict between existing resources and one that the operators deploy (for instance aws/eks-charts/issues/57)

[morganchristiansson]

There is open issue for EKS CNI addon to allow configuration values aws/containers-roadmap#1333 (please vote by giving it a 👍)

As the EKS CNI addon doesn't support any configuration I am using the helm chart https://github.com/aws/eks-charts/tree/master/stable/aws-vpc-cni

[jayanthvn]

@jackivanov As @morganchristiansson suggested helm chart would be a good option.

Even with managed add ons, we need to have cluster to be existing.

[jayanthvn]

Closing the issue now. Please feel free to reach out for any further assistance.