Adding eks-customer module #731

andrleite · 2024-09-10T00:32:44Z

Summary

This PR will add a new module to create EKS clusters + deploy utilities using Argocd.

Ticket Link

https://mattermost.atlassian.net/browse/CLD-7696

Release Note

New eks-customer module

aws/eks-customer/eks.tf

stafot · 2024-09-18T07:15:59Z

aws/eks-customer/eks.tf

@@ -0,0 +1,101 @@
+module "eks" {
+  source  = "terraform-aws-modules/eks/aws"
+  version = "20.14.0"


lets make version variable

I believe this isn't possible due to terraform limitations. hashicorp/terraform#28912

aws/eks-customer/eks.tf

aws/eks-customer/eks_managed_node-group.tf

stafot · 2024-09-18T07:18:55Z

aws/eks-customer/irsa.tf

+      namespace_service_accounts = ["${each.value.name}:${each.value.service_account}"]
+    }
+  }
+}


Great work!

stafot · 2024-09-18T07:20:25Z

aws/eks-customer/scripts/utils.sh

+    while_repo_exists
+    git clone "https://${GIT_REPO_USERNAME}:${GITLAB_OAUTH_TOKEN}@${GIT_HOST}/${GIT_REPO_PATH}" $gitops_sre_dir
+    git config --global user.name "${GIT_REPO_USERNAME}"
+    git config --global user.email "${GIT_REPO_USERNAME}@mattermost.com"


make it $GIT_REPO_URL

aws/eks-customer/utility_policies.tf

aws/eks-customer/versions.tf

aws/eks-customer/example/main.tf

angeloskyratzakos · 2024-09-18T12:16:48Z

aws/eks-customer/irsa.tf

+
+module "irsa" {
+  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
+  version = "5.44.0"


this version should be hardcoded?

No, let's make it variable Thanks

I believe this isn't possible due to terraform limitations. hashicorp/terraform#28912

angeloskyratzakos · 2024-09-18T12:21:23Z

aws/eks-customer/scripts/deploy-utility.sh

@@ -0,0 +1,132 @@
+#/usr/bin/env bash


Suggested change

#/usr/bin/env bash

#!/usr/bin/env bash

angeloskyratzakos · 2024-09-18T12:24:55Z

aws/eks-customer/scripts/deploy-utility.sh

+
+function create_cluster_folder() {
+	echo "Creating custom values file"
+	mkdir $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME} || true


|| true suppresses all errors, was there any reason for that?

This is to avoid errors when the file already exists.

angeloskyratzakos · 2024-09-18T12:35:08Z

aws/eks-customer/scripts/deploy-utility.sh

+    sed -i "s/<CLUSTER_ID>/${CLUSTER_NAME}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<ENV>/${ENV}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<CLUSTER_ID>/${CLUSTER_NAME}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<CERTFICATE_ARN>/$certificate_arn/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<PRIVATE_CERTIFICATE_ARN>/$private_certificate_arn/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<VPC_ID>/${VPC_ID}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<PRIVATE_DOMAIN>/${PRIVATE_DOMAIN}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<IP_RANGE>/$allow_list_cidr_range/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/hostNetwork: false/hostNetwork: true/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml


maybe something

sed -i -e "s/<CLUSTER_ID>/${CLUSTER_NAME}/g" \ -e "s/<ENV>/${ENV}/g" \ -e "s/<CERTIFICATE_ARN>/$certificate_arn/g" \ ... \ "$target_file"

since the target file is the same

angeloskyratzakos · 2024-09-18T12:36:11Z

aws/eks-customer/scripts/deploy-utility.sh

+    sed -i "s/<CLUSTER_ID>/${CLUSTER_NAME}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<ENV>/${ENV}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<CLUSTER_ID>/${CLUSTER_NAME}/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml
+    sed -i "s/<CERTFICATE_ARN>/$certificate_arn/g" $gitops_apps_dir/${ENV}/helm-values/${CLUSTER_NAME}/$utility_name-custom-values.yaml


there is a typo: CERTFICATE_ARN maybe CERTIFICATE_ARN

angeloskyratzakos · 2024-09-18T12:38:20Z

aws/eks-customer/scripts/remove-utility.sh

+  clone_repo
+  remove_utilities
+  remove_helm_values
+  wait_for_argocd
+  remove_cluster
+  clean_up


clone_repo and clean_up funcs exist in other script, will they be available in that main?

yes, because I'm "loading" them into the script

source $(dirname "$0")/utils.sh

angeloskyratzakos · 2024-09-18T12:38:57Z

aws/eks-customer/scripts/deploy-utility.sh

+  clone_repo
+  add_cluster
+  create_cluster_folder
+  deploy_utility
+  clean_up


clone_repo and clean_up funcs exist in other script, will they be available in that main?

yes, because I'm "loading" them into the script

source $(dirname "$0")/utils.sh

angeloskyratzakos · 2024-09-18T12:39:48Z

aws/eks-customer/scripts/remove-utility.sh

@@ -0,0 +1,47 @@
+#/usr/bin/env bash


Suggested change

#/usr/bin/env bash

#!/usr/bin/env bash

angeloskyratzakos · 2024-09-18T12:40:00Z

aws/eks-customer/scripts/utils.sh

@@ -0,0 +1,88 @@
+#/usr/bin/env bash


Suggested change

#/usr/bin/env bash

#!/usr/bin/env bash

aws/eks-customer/utility_policies.tf

aws/eks-customer/utility-utils.tf

angeloskyratzakos · 2024-09-18T12:59:50Z

aws/eks-customer/scripts/remove-utility.sh

+        echo "Commiting changes: Adding cluster ${CLUSTER_NAME}"
+        commit_changes "Remove utilities: ${CLUSTER_NAME}" $gitops_apps_dir/${ENV}/helm-values
+    fi
+    echo "No helm values found for cluster ${CLUSTER_NAME}"


This message is printed regardless of whether the directory exists or not.

Suggested change

echo "No helm values found for cluster ${CLUSTER_NAME}"

else

echo "No helm values found for cluster ${CLUSTER_NAME}"

fi

not sure if that's better

angeloskyratzakos · 2024-09-18T13:03:54Z

aws/eks-customer/scripts/remove-utility.sh

+}
+
+function remove_cluster() {
+    local cluster_id=$1


is that used?

nope! good catch. old value.

andrleite added 30 commits August 23, 2024 17:56

[CLD-7969]Initial commit for eks-customer module

85169d5

[CLD-7969]change source reference

080202a

[CLD-7969]refactoring eks customer module

c72bc66

[CLD-7969]wip: deploy utility

b3120a3

CLD-7696: add utilities module

47adfeb

[CLD-7696]Initial commit for eks-customer module

cb8a10b

[CLD-7696]remove unused line

4603287

[CLd-7697]Adding utillites scripts

78081a5

[CLd-7697]Unify utilities in a single module to handle destroy order

74fa4d6

[CLD-7696]Add destroy functions

a402e0a

[CLD-7696]Ignore terraform-lock

19e4e05

[CLD-7696]Fix lint

5d5df5d

[CLD-7696]Remove argocd-utility module

0c7a942

[CLD-7696]Remove argocd-utility module

a5218f2

[CLD-7696]Add wait function to prevent git conflict

e6c76f7

[CLD-7696]update readme

9a5a9a7

[CLD-7696]Fix dependencies

cd6efb6

[CLD-7696]Fix data dependency

6525ac7

[CLD-7696]Fix EOL

46ad323

[CLD-7696]remove old block

b00dcbc

[CLD-7696]fix lint

0450d9d

[CLD-7696]update readme

2c457d7

[CLD-7696]use module outout instead of variable for cluster name

300b069

[CLD-7696]Add provisioner role for cluster access

d8348c0

[CLD-7696]adding random id for cluster name

154715e

[CLD-7696]use module outout instead of variable for cluster name

7190c4b

[CLD-7696]check if gitops repo exists; get repo latest changes

9eca24b

[CLD-7696]change tf version to match with atlantis

923832b

[CLD-7696]add git host keys to know_hosts

14891e3

[CLD-7696]fix git_host var when destroy

5339737

andrleite requested a review from a team as a code owner September 17, 2024 20:56

stafot requested changes Sep 18, 2024

View reviewed changes

angeloskyratzakos reviewed Sep 18, 2024

View reviewed changes

aws/eks-customer/example/main.tf Show resolved Hide resolved

angeloskyratzakos reviewed Sep 18, 2024

View reviewed changes

aws/eks-customer/utility_policies.tf Outdated Show resolved Hide resolved

angeloskyratzakos reviewed Sep 18, 2024

View reviewed changes

aws/eks-customer/utility-utils.tf Outdated Show resolved Hide resolved

angeloskyratzakos reviewed Sep 18, 2024

View reviewed changes

andrleite added 8 commits September 18, 2024 11:39

[CLD-7696]Make hardcoded values as variables

081d8e1

[CLD-7696]Fix git url variable name

f1318a6

[CLD-7696]Fix hardcoded envrionments

4952890

[CLD-7696]Remove unused provider

949948b

[CLD-7696]Minor improvements to bash scripts

e2b52ef

[CLD-7696]Fix modules variables

71b8f6d

[CLD-7696]Remove duplicated variable

d61a764

[CLD-7696]Fix sed cmd

17c9a9b

andrleite requested review from angeloskyratzakos and stafot September 18, 2024 19:26

[CLD-7696]Fix example

2c0e5cc

stafot approved these changes Sep 19, 2024

View reviewed changes

andrleite merged commit 77f3669 into master Sep 20, 2024
1 check passed

andrleite deleted the cld-7696 branch September 20, 2024 19:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding eks-customer module #731

Adding eks-customer module #731

andrleite commented Sep 10, 2024

stafot Sep 18, 2024

andrleite Sep 18, 2024

stafot Sep 18, 2024

stafot Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

andrleite Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

angeloskyratzakos Sep 18, 2024

andrleite Sep 18, 2024

Adding eks-customer module #731

Adding eks-customer module #731

Conversation

andrleite commented Sep 10, 2024

Summary

Ticket Link

Release Note

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment