Automated cluster setup for AWS

[yuvipanda]

Description

We use eksctl for setting up AWS clusters. The setup isn't fully automated yet, however. We should use a combination of documentation + any required helper scripts to automate the entire setup.

Benefit

Automating our AWS clusters will allow us to standardize and automate our infrastructure, which is an important part of scaling our infrastructure and ensuring that it behaves consistently across all of our clusters.

Tasks to complete

• Guidelines for using kops vs EKS #431
• Use an AWS IAM Role / User to fetch kubernetes credentials in our deployer #381
• Convert AWS cluster deployment workflow to eksctl #737

Port over this code for authentication:
https://github.com/yuvipanda/hubploy/blob/dfc00254abb30d8887230472e83b405ff5872933/hubploy/auth.py#L107

[choldgraf]

I am a bit confused about the state of our AWS cluster setup. In this description it says:

We use eksctl for setting up AWS clusters

but our pilot hubs documentation is all about kops rather than eksctl (https://devops.2i2c.org/en/latest/howto/operate/new-cluster/aws.html). Is the plan to move away from kops in general, in favor of eksctl? (I am definitely in favor of only using 1 pathway instead of 2)

Either way, I've opened up #737 and added it above as a step towards completing this

[damianavila]

Just to clarify...

We use kops for Openscapes and Farallon clusters AND we use EKS (and eksctl) for the Carbonplan cluster.
There is an ongoing ticket to compare both at #431.

On a brief comment in 737, I have said why I believe we should keep both for a little bit until we collect enough information to decide and resolve #431.

[yuvipanda]

This is done now - we have https://github.com/2i2c-org/infrastructure/blob/master/docs/howto/operate/new-cluster/aws.md. eksctl creates the cluster, and we have terraform for all other work. I'd say we can close this for now.

The next step would be to remove eksctl completely, but I think that requires aws/containers-roadmap#724 to be solved.