Skip to content

Security Improvement's associated with Jira Task CSI-1647 #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion config/eks/higher_envs_eks_config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ export function apply_config(config: Easy_EKS_Config_Data, stack: cdk.Stack){ //
config.setBaselineMNGType(eks.CapacityType.ON_DEMAND);
if(process.env.CDK_DEFAULT_ACCOUNT==="111122223333"){
config.addClusterAdminARN(`arn:aws:iam::111122223333:user/example`);
/* Note:
/* Note 1:
The IAM user/role running cdk deploy dev1-eks, gets added to the list of Cluster Admins by default.
This is done for convenience, if you want to change this default, you'll need to edit ./lib/Easy_EKS.ts

Note 2:
config.addClusterAdminARN('...:user/example') should only be used in an if statement,
Because the identity referenced in ARN must exist or the deployment will fail
This allows you to create a explicit list of ARNs (representing IAM roles or users)
Expand Down
8 changes: 6 additions & 2 deletions config/eks/lower_envs_eks_config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@ export function apply_config(config: Easy_EKS_Config_Data, stack: cdk.Stack){ //
config.setBaselineMNGSize(2);
config.setBaselineMNGType(eks.CapacityType.SPOT);
if(process.env.CDK_DEFAULT_ACCOUNT==="111122223333"){
config.addClusterAdminARN(`arn:aws:iam::111122223333:user/example`);
/* Note:
config.addClusterAdminARN(`arn:aws:iam::111122223333:user/example`);
/* Note 1:
The IAM user/role running cdk deploy dev1-eks, gets added to the list of Cluster Admins by default.
This is done for convenience, if you want to change this default, you'll need to edit ./lib/Easy_EKS.ts

Note 2:
config.addClusterAdminARN('...:user/example') should only be used in an if statement,
Because the identity referenced in ARN must exist or the deployment will fail
This allows you to create a explicit list of ARNs (representing IAM roles or users)
Expand Down
8 changes: 4 additions & 4 deletions docs/03_Quickstart/Quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,10 +299,10 @@ dev1-eks: creating CloudFormation changeset...
✨ Deployment time: 1052.74s

Outputs:
dev1-eks.dev1eksConfigCommand9B300592 = aws eks update-kubeconfig --name dev1-eks --region ca-central-1 --role-arn arn:aws:iam::905418347382:role/dev1-eks-assumableEKSAdminAccessRoleC284FA0F-F22KjKQrjLpO
dev1-eks.dev1eksGetTokenCommandDE6D6947 = aws eks get-token --cluster-name dev1-eks --region ca-central-1 --role-arn arn:aws:iam::905418347382:role/dev1-eks-assumableEKSAdminAccessRoleC284FA0F-F22KjKQrjLpO
Stack ARN:
arn:aws:cloudformation:ca-central-1:905418347382:stack/dev1-eks/60d97430-27d0-11f0-ae18-0e71104456e5
dev1-eks.iamWhitelistedKubeConfigCmd = aws eks update-kubeconfig --region ca-central-1 --name dev1-eks

Note: This only works for the user/role deploying cdk and IAM Whitelisted Admins.
To learn more review ./easyeks/config/eks/lower_envs_eks_config.ts

✨ Total time: 1069.52s
```
Expand Down
81 changes: 54 additions & 27 deletions lib/Easy_EKS.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
// The point of this is to investigate not using eks-blueprints, which has been problematic.
// * not as maintained as I 1st thought, https://github.com/aws-quickstart/cdk-eks-blueprints/graphs/code-frequency
// * It's karpenter has been busted for a long time, I suspect this standalone could be more reliable https://github.com/aws-samples/cdk-eks-karpenter
// * Many operations can only be done against object of type eks.Cluster, blueprints made that hard to get access to
// I found edge case limitations where even though I can access it by overriding an internal protected method I can't do things like deploy yaml
// due to an order of operations issue, so going to do a major refactor to get rid of EKS Blueprint in favor of L2 construct.
// v2 will be based on https://github.com/aws/aws-cdk/tree/main/packages/aws-cdk-lib/aws-eks#aws-iam-mapping

import console = require('console'); //can help debug feedback loop, allows `console.log("hi");` to work, when `cdk list` is run.
import { Construct, IConstruct } from 'constructs';
import * as cdk from 'aws-cdk-lib';
Expand All @@ -24,11 +16,36 @@ import * as test_eks_config from '../config/eks/test_eks_config';
import * as stage_eks_config from '../config/eks/stage_eks_config';
import * as prod_eks_config from '../config/eks/prod_eks_config';
import * as observability from './Frugal_GPL_Observability_Stack';
import { execSync } from 'child_process'; //work around for kms UX issue
import * as shell from 'shelljs'; //npm install shelljs && npm i --save-dev @types/shelljs
import request from 'sync-request-curl'; //npm install sync-request-curl (cdk requires sync functions, async not allowed)



class CDK_Deployer {
// After cdk deploy dev1-eks is run, a kubectl population command will be displayed
// Example:
// aws eks update-kubeconfig --region ca-central-1 --name dev1-eks
//
// For good security we lock this down to whitelisted IAM access entries, defined in the Access tab of EKS's web console
// For convienence we make an assumption that the IAM identity running cdk deploy dev1-eks, should be auto-added to that list.
// A singleton pattern is used to avoid multiple lookups.
private static singleton: CDK_Deployer;
private static iam_id: string;
public static get_IAM_ID(): string{
if(!CDK_Deployer.singleton){
CDK_Deployer.singleton = new CDK_Deployer();
const cmd = `aws sts get-caller-identity | jq .Arn | tr -d '"|\n|\r'`; //translate delete (remove) double quote & new lines
const cmd_results = shell.exec(cmd, {silent:true});
if(cmd_results.code===0){
this.iam_id = cmd_results.stdout; //plausible value = arn:aws:iam::111122223333:user/example
}
}
return CDK_Deployer.iam_id; //returns IAM of user/role running `cdk deploy dev1-eks`
}
} //end CDK_Deployer



export class Easy_EKS{ //purposefully don't extend stack, to implement builder pattern and give more flexibility for imperative logic.

//Class Variables/Properties:
Expand Down Expand Up @@ -227,13 +244,7 @@ export class Easy_EKS{ //purposefully don't extend stack, to implement builder p
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//Logic to create cluster:
//UX convienence similar to EKS Blueprints
const assumableEKSAdminAccessRole = new iam.Role(this.stack, 'assumableEKSAdminAccessRole', {
assumedBy: new iam.AccountRootPrincipal(), //<-- root as is root of the account,
// so assumable by any principle/identity in the account.
});

// Logic to create cluster:
// if (kms.Key.isLookupDummy(kms.Key.fromLookup(this.stack, "pre-existing-kms-key", { aliasName: this.config.kmsKeyAlias, returnDummyKeyOnMissing: true, }))){
// eksBlueprint.resourceProvider(blueprints.GlobalResources.KmsKey, new blueprints.CreateKmsKeyProvider(
// this.config.kmsKeyAlias, {description: "Easy EKS generated kms key, used to encrypt etcd and ebs-csi-driver provisioned volumes"}
Expand All @@ -251,9 +262,16 @@ export class Easy_EKS{ //purposefully don't extend stack, to implement builder p
defaultCapacity: 0,
tags: this.config.tags,
authenticationMode: eks.AuthenticationMode.API_AND_CONFIG_MAP,
mastersRole: assumableEKSAdminAccessRole, //<-- adds aws eks update-kubeconfig output
// mastersRole: //<-- This is the built in way to set output associated with update-kubeconfig
// The role specified is usually assumable by iam.AccountRootPrinciple which isn't secure
// so mastersRole is purposefully commented out / not implemented for improved security.
secretsEncryptionKey: kms_key,
});
//v-- This achieve similar yet better results than commented out mastersRole
const msg = `aws eks update-kubeconfig --region ${this.stack.region} --name ${this.config.id}\n\n` +
'Note: This only works for the user/role deploying cdk and IAM Whitelisted Admins.\n'+
' To learn more review ./easyeks/config/eks/lower_envs_eks_config.ts';
const output_msg = new cdk.CfnOutput(this.stack, 'iamWhitelistedKubeConfigCmd', { value: msg });
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Expand Down Expand Up @@ -313,6 +331,13 @@ export class Easy_EKS{ //purposefully don't extend stack, to implement builder p
*/
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Logic to add the person running cdk deploy to the list of cluster admins
// This satisfies EKS IAM access entry rights prerequisite, needed to allow the output command to work
// aws eks update-kubeconfig --region ca-central-1 --name dev1-eks
this.config.addClusterAdminARN(CDK_Deployer.get_IAM_ID());
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Logic to add Cluster Admins Defined in Config as Code:
const clusterAdminAccessPolicy: eks.AccessPolicy = eks.AccessPolicy.fromAccessPolicyName('AmazonEKSClusterAdminPolicy', {
Expand Down Expand Up @@ -368,7 +393,10 @@ function initialize_WorkerNodeRole(stack: cdk.Stack){
},
});
EKS_Worker_Node_Role.applyRemovalPolicy(cdk.RemovalPolicy.RETAIN); //Workaround to avoid cdk destroy bug
return EKS_Worker_Node_Role
// ^--This leaves orphaned IAM roles, which isn't ideal, yet least bad option.
// Shouldn't hurt anything in most cases (there is a max of 1000 IAM roles per AWS account)
// It'd only be a problem after 100's of deploy & destroy operations, after which it's easy to manually clean up.
return EKS_Worker_Node_Role;
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Expand Down Expand Up @@ -647,16 +675,15 @@ function ensure_existance_of_aliased_kms_key(kmsKeyAlias: string, stackName: str
* staging envs share a kms key: "alias/eks/stage"
* prod envs share a kms key: "alias/eks/prod"
*/
let kms_key:kms.Key;
const cmd = `aws kms list-aliases --region ${region} | jq '.Aliases[] | select(.AliasName == "${kmsKeyAlias}") | .TargetKeyId'`
const cmd_results = execSync(cmd).toString();
const cmd_results = shell.exec(cmd,{silent:true}).stdout;
let key_id = "";
if(cmd_results===""){ //if alias not found, then make a kms key with the alias
const create_key_cmd = `aws kms create-key --region ${region} --description="Easy EKS generated kms key, used to encrypt etcd and ebs-csi-driver provisioned volumes"`
const results = JSON.parse( execSync(create_key_cmd).toString() );
const results = JSON.parse( shell.exec(create_key_cmd,{silent:true}).stdout );
key_id = results.KeyMetadata.KeyId;
const add_alias_cmd = `aws kms create-alias --alias-name ${kmsKeyAlias} --target-key-id ${key_id} --region ${region}`;
execSync(add_alias_cmd);
shell.exec(add_alias_cmd,{silent:true});
//get the ebs csi role, so it can be used to add permissions to the new key
}
// disabled for now, as we need to test that it assigns the permissions correctly before enable customer eks
Expand All @@ -672,8 +699,8 @@ function ensure_existance_of_aliased_kms_key(kmsKeyAlias: string, stackName: str
/*function give_kms_access_to_ebs_csi_role(stackName: string, region: string, KeyId: string){
const roleName = stackName + '-awsebscsidriveriamrole';
const cdm_list_ebs_csi_role = `aws iam list-roles --query "Roles[?contains(RoleName, '${roleName}')].Arn" --output text`;
const list_roles = execSync(cdm_list_ebs_csi_role);
if (list_roles.toString() !== '') {
const list_roles_cmd = shell.exec(cdm_list_ebs_csi_role,{silent:true}).stdout;
if (list_roles_cmd !== '') {
const policy = `{
"Version": "2012-10-17",
"Id": "key-default-1",
Expand All @@ -691,7 +718,7 @@ function ensure_existance_of_aliased_kms_key(kmsKeyAlias: string, stackName: str
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "${list_roles.toString().trim()}"
"AWS": "${list_roles_cmd.trim()}"
},
"Action": [
"kms:Encrypt",
Expand All @@ -707,8 +734,8 @@ function ensure_existance_of_aliased_kms_key(kmsKeyAlias: string, stackName: str
}
]
}`;
const cmp_policy = `aws kms put-key-policy --policy-name default --key-id ${KeyId.trim()} --region ${region} --policy '${policy}'`;
execSync(cmp_policy);
const cmp_policy_cmd = `aws kms put-key-policy --policy-name default --key-id ${KeyId.trim()} --region ${region} --policy '${policy}'`;
shell.exec(cmp_policy_cmd,{silent:true}).stdout;
} else {
console.log(`EBS CSI Role with name: ${roleName} already exists.`);
}
Expand Down
20 changes: 14 additions & 6 deletions lib/Easy_EKS_Config_Data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import * as eks from 'aws-cdk-lib/aws-eks';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as kms from 'aws-cdk-lib/aws-kms';
import { execSync } from 'child_process';
import * as shell from 'shelljs'; //npm install shelljs && npm i --save-dev @types/shelljs
import { validateTag } from './Utilities';

export class Easy_EKS_Config_Data { //This object just holds config data.
Expand Down Expand Up @@ -59,7 +59,7 @@ export class Easy_EKS_Config_Data { //This object just holds config data.
// knowledge of how to handle the edge case.
////////////////////////////////////////////////////////////////////////////////
const cmd = `aws ec2 describe-vpcs --filter Name=tag:Name,Values=${vpcName} --query "Vpcs[].VpcId" | tr -d '\r\n []"'`
const cmd_results = execSync(cmd).toString();
const cmd_results = shell.exec(cmd, {silent:true}).stdout;
// Plausible Values to expect:
// case 1: cmd_results===""
// ^-- Occurs when vpc name not found. Would cause an error, which is desired behavior as we assume it should be found.
Expand All @@ -86,7 +86,7 @@ export class Easy_EKS_Config_Data { //This object just holds config data.
try {
validateTag(key, value)
if(this.tags === undefined){ this.tags = { [key] : value } }
else{ this.tags = { ...this.tags, [key] : value }}
else{ this.tags = { ...this.tags, [key] : value } }
} catch (error: any) {
console.error("Error:", error.message)
throw "Error validating tags. See details above"// Using throw here to stop the checks; otherwise an error will print out for every place this tag would be applied, and the process will continue
Expand All @@ -100,12 +100,20 @@ export class Easy_EKS_Config_Data { //This object just holds config data.
}
setIpMode(ipMode: eks.IpFamily){ this.ipMode = ipMode; }
addClusterAdminARN(arn:string){ //v--initialize if undefined
if(this.clusterAdminAccessEksApiArns === undefined){ this.clusterAdminAccessEksApiArns = [arn] }
else{ this.clusterAdminAccessEksApiArns.push(arn); } //push means add to end of array
if(this.clusterAdminAccessEksApiArns === undefined){ this.clusterAdminAccessEksApiArns = [arn]; }
else{
if(!this.clusterAdminAccessEksApiArns.includes(arn)){ //if value isn't already in array
this.clusterAdminAccessEksApiArns.push(arn); //then push, as in add, to end of array
}
}
}
addClusterViewerAccount(account:string){ //v--initialize if undefined
if(this.clusterViewerAccessAwsAuthConfigmapAccounts === undefined){ this.clusterViewerAccessAwsAuthConfigmapAccounts = [account] }
else{ this.clusterViewerAccessAwsAuthConfigmapAccounts.push(account); } //push means add to end of array
else{
if(!this.clusterViewerAccessAwsAuthConfigmapAccounts.includes(account)){ //if value isn't already in array
this.clusterViewerAccessAwsAuthConfigmapAccounts.push(account); //then push, as in add, to end of array
}
}
}
setKmsKeyAlias(kms_key_alias: string){
/*Note (About UX improvement logic):
Expand Down
Loading