Automatically deploy carbonplan hub

[yuvipanda]

Description

Currently when you change things in pilot-hubs, it doesn't actually update CarbonPlan's hub. We should build auto-deployment infrastructure instead.

Was tried in yuvipanda@f687264 but reverted, as the auth provided by the checked-in kubeconfig is not enough. We need #381 with eks support to be able to deploy automatically from CI

Tasks

• Get @damianavila access to work on this hub
• Create an AWS user with credentials setup with access to the EKS cluster
• Add support to our deployer to fetch k8s credentials from eks (porting this code)
• Deploy changes whenever a change is merged here

[yuvipanda]

@damianavila what's your carbonplan AWS username? I can give you access to the EKS cluster by following https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html

[damianavila]

@damianavila what's your carbonplan AWS username?

I have pinged you in Slack with that info 😉 .

[yuvipanda]

@damianavila I added you to list of users who can access the EKS cluster, with

eksctl create iamidentitymapping -f carbonplan.yaml --group system:masters --username admin --arn arn:aws:iam::631969445205:user/damian

[choldgraf]

Does this issue depend on #627 being resolved? It feels like this is not possible until we have the infrastructure to automatically deploy AWS hubs.

[damianavila]

It depends but it is not a blocker to make progress on this one...

We can decouple this issue (hub deployment automation) from the #627 issue (cluster deployment automation) if we manually add the "2i2c deployer user" to the existing EKS carbonplan cluster as @yuvipanda did with my user in his previous comment.

In fact, the approach I am following is essentially impersonating 😜 the "2i2c deployer user" with my credentials. Eventually, we should be able to manually create the real "2i2c deployer user", manually get its credentials and manually add it to the cluster and when #627 is ready, all of these steps should be done automatically.

Unless I am missing something, which is usually the case 😉 .

[damianavila]

Add support to our deployer to fetch k8s credentials from eks (porting this code)

Update: Draft PR #636 tries to deal with this item.

[damianavila]

Update:

• Created a deployer user in the AWS carbonplan space under the 2i2c group.
• Added the identity mapping for that new deployer user so it can access the EKS cluster
• Got access keys for that user and encrypted them

Tested manually (after some adjusting in the logic) and it seems to be working as expected.

[damianavila]

Some commands I have used for the above update:

• Create a new user:
  aws iam create-user --user-name deployer

• Add the user to the 2i2c group
  aws iam add-user-to-group --group-name 2i2c --user-name deployer

• Check if the user was added to the group
  aws iam get-group --group-name 2i2c

• Get the credentials:
  aws iam create-access-key --user-name deployer | tee /tmp/deployer.json

• Add the user to the EKS cluster
  eksctl create iamidentitymapping --cluster carbonplanhub --region=us-west-2 --arn arn:aws:iam::631969445205:user/deployer --group system:masters --username admin

• Check if the user was added to the map
  eksctl get iamidentitymapping --cluster carbonplanhub --region=us-west-2

[damianavila]

For posterity, this one was deployed successfully after merging it: https://github.com/2i2c-org/pilot-hubs/runs/3533569179?check_suite_focus=true