Use an AWS deployer user to fetch kubernetes credentials in our deployer #673
Conversation
We currently manage EKS and kops clusters on AWS land. With this commit we are adding support and validation (by jsonschema) for the kops-based cluster that requires the state store to properly get the state of the cluster and succeed in the next step at the time to get export the kubeconfig. Notice I am using the if and then keywords [1] to require the stateStore property only if the kops value is configured. The encrypted key file referenced in the condfig file will be provided in a later commit. [1] https://json-schema.org/understanding-json-schema/reference/conditionals.html#if-then-else
We identified a 2i2c-engineers groups in Farallon AWS and we have created a deployer user under that group. We used common awscli commands to perform this task and retrieve the credentials. More details in [1]. Finally, we encrypted the file with sops accordingly with the current established workflows to manage secret files. [1] #381 (comment)
Previously, we have added an auth_aws function to retrieve the kubeconfig and connect to EKS-based cluster. With this commit, we are adding a new subprocess "brach" to conditionally `kops export` the kubeconfig (if the kops option is configured) and get access to kops-based clusters, such as the Farallon one.
I am still not sure how the action performs for real but it is the only one available in the marketplace and we should probably give it a try. I have also added a commented openscapes line for the future ;-)
|
Thanks for the feedback, @GeorgianaElena! Btw, local testing on top of staging seems to be working as expected: Also, run successfully with the test enabled 😉 |
|
Tested the kops action in a test repo and it is not working 😢 |
|
@GeorgianaElena and @sgibson91, I pushed another commit that I think you would be interested in. |
|
@damianavila Looks good to me! That's a totally valid workflow step :) |
|
The only thing we might want to do is, instead of always pulling the latest release of |
|
@sgibson91, I think the last commit addressed what you ask for, let me know if that is not the case. |
|
OK, I will proceed with the merge since I have at least one approval and nobody is working in the prod hub now. |
|
And it obviously failed 😛 |
|
@sgibson91, I am surprised this is failing because, accordingly to the docs that should work
unless I am missing something, which is obviously the case 😉 |
|
Well... testing the same on my test repo works as expected... mmm... https://github.com/damianavila/test/blob/main/.github/workflows/test_kops_gha.yml |
|
So, it seems there is some permission error preventing curl to write the file on this repo... wondering why the behavior is different from my repo. |
|
Hmmm could it be that we have run into https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/ ? I believe @consideRatio had set the default permissions of all repos to be read only |
|
@choldgraf i would agree except GITHUB_TOKEN isn't being invoked 🤔 |
|
This could be a closed pipe issue and I suspect that if we reran CI, it would be a flaky test and pass sometimes and not others |
|
@choldgraf no change made by me regarding that, but I suggest that permissions for GITHUB_TOKEN are explicitly specified - especially in any templates passed to others, so our org and other orgs can narrow down permissions as a security measure and still have the workflows functional. This is #359 |
|
I think this is curl-specific, I will switch to wget and try again 😉 |
|
For posterity, this was not a curl nor a wget issue. |
Summary:
The auth_aws function I recently added to the deployer is now authenticating kops-based clusters.
Farallon would be the first one.
You can see more details in the commit messages.
You may find interesting the usage of if and else keyword in the schema to properly validate the config file.
Btw, this one still needs to be more tested but I believe is complete so putting it out there to get early feedback.
Closes #381