Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit list of tag required #38

Closed
lra opened this issue Apr 9, 2019 · 9 comments
Closed

Explicit list of tag required #38

lra opened this issue Apr 9, 2019 · 9 comments
Assignees

Comments

@lra
Copy link

lra commented Apr 9, 2019

Hi,

While trying to build a EKS cluster without using CloudFormation, I had to do some trial and error to figure out the tags required by a fully working EKS cluster.

Should the doc include a specific page stating all the required tags for people not using CF? Am I missing some btw?

Something like this (I can do a PR if needed):

Tags required by EKS

VPC Tagging Requirement

  • kubernetes.io/cluster/<cluster-name> set to shared.

See https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html.

Subnet Tagging Requirement

  • kubernetes.io/cluster/<cluster-name> set to shared.
  • For private subnets: kubernetes.io/role/internal-elb set to 1.
  • For public subnets: kubernetes.io/role/elb set to 1.

See https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html.

Security Group Tagging Requirement

The security group used by the worker nodes should be tagged with:

  • kubernetes.io/cluster/<cluster-name> set to owned

See https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2019-02-11/amazon-eks-nodegroup.yaml.

Autoscaling Group Tagging Requirement

The Autoscaling group used to launch the worker nodes should be tagged with:

  • kubernetes.io/cluster/<cluster-name> set to owned

Also set PropagateAtLaunch = true to copy the tag to EC2 instances that are
launched as part of the Auto Scaling group

See https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2019-02-11/amazon-eks-nodegroup.yaml.

@vamshi-sfdc
Copy link

@nrdlngr can you please advise

@nrdlngr
Copy link

nrdlngr commented Aug 21, 2019

Sorry for the delayed response. I think my best advice would be for most people to use eksctl or our AWS CloudFormation templates to create clusters, because they take care of these tags for you automatically.

Can you help me understand why you would want to create a cluster without using one of these easier methods so I can understand your use case?

I can certainly add the AutoScaling group tagging requirements, but I'll have to figure out the best spot for them. And I can see that having all of the basic tagging requirements in one place would be beneficial, although other add-ons might have tagging requirements that would not necessarily be listed there.

@lra
Copy link
Author

lra commented Aug 22, 2019

Our use case was to build a terraform module to provision EKS clusters and its resources (VPCs, subnets, datastores, and such) from scratch.

To build such a module, you have to know what the end state is.

We use terraform (and not cloudformation) and I had to trial & error and look into the source of kops and k8s to figure out the needed tags and what they meant. I just feel it might be worthwhile to document for those who want to use EKS without CF or eksctl, that might lead to more use for EKS in the end.

eksctl is not covering the same needs and forcing some technologies (e.g. cloudformation) that might not work for some, terraform is totally agnostic and transparent about what it does.

We got the module now, with a few clusters using it, so LMK if I can help.

@nrdlngr
Copy link

nrdlngr commented Aug 22, 2019

OK, that makes sense. I think adding a topic with the required tags for basic cluster creation would be valuable for our customers. I'll add this work item to my backlog, and I'll update the issue when the content is live.

Thanks for helping us to improve our documentation!

@ashwanijha04
Copy link

ashwanijha04 commented Jan 26, 2020

EC2 Tagging requirements:

- k8s.io/cluster-autoscaler/<cluster_name> = owned
- k8s.io/cluster-autoscaler/enabled = true 
- kubernetes.io/cluster/<cluster_name> = owned 

@kahirokunn
Copy link

kahirokunn commented Jan 18, 2022

Sorry. This is the wrong place to comment.

@duckie
Copy link

duckie commented Mar 20, 2022

Technically, only k8s.io/cluster-autoscaler/enabled = true is required for auto-scaling, you just need to properly instruct the cluster autoscaler to use kubernetes.io/cluster/<cluster_name> = owned and it will be fine.

Honestly I dont get why AWS do not publish a properly curated cluster autoscaler conf for EKS. I mean, c'mon guys, you give a list of instructions on how to patch it. Just script the patching and share the result. Will save users some time and headaches and your support some trouble.

@macdrorepo
Copy link

Did anyone encountered issue that after deployment of ingress / ALB required tags disappearing from public subnets where ALB is deployed?

@jimdial-aws
Copy link

Closing this due to its age. The tagging requirements change frequently for different resources and the versions you're using. The docs are updated regularly with the current tagging requirements for different resources.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants