Added terraform modules for cluster and network, eks using them and keda #648

juanmatias · 2024-10-31T18:39:11Z

What?

Extracted the network and cluster layers as modules
network can be used for base-network and for EKS networks
cluster can be used to create EKS cluster in a simpler way
Added keda to components

Important

Note cluster and network sublayers were propagated to demoapps and base clusters. Since their networks are still in place, and to avoid importing, it is recommended to destroy those networks before this PR is merged.

Why?

To promote DRY
To have KEDA under the scope

References

Related Documentation in this PR Added documentation on EKS and KEDA le-ref-architecture-doc#232

github-actions · 2024-10-31T18:40:19Z

💰 Infracost report

Monthly estimate decreased by $366 📉

Changed project	Baseline cost	Usage cost*	Total change	New monthly cost
binbashar/le-tf-infra-aws/apps-...ast-1/k8s-eks-demoapps/cluster	+$1	-	+$1 (+1%)	$106
binbashar/le-tf-infra-aws/apps-...ast-1/k8s-eks-demoapps/network	+$33	-	+$33	$33
binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/cluster	-$432	-	-$432 (-80%)	$106
binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/network	+$33	-	+$33	$33

*Usage costs can be estimated by updating Infracost Cloud settings, see docs for other options.

Estimate details (includes details of unsupported resources and skipped projects due to errors)

Key: * usage cost, ~ changed, + added, - removed

──────────────────────────────────
Project: apps-devstg-us-east-1-k8s-eks-demoapps-cluster
Module path: apps-devstg/us-east-1/k8s-eks-demoapps/cluster

+ module.eks.module.cluster.aws_eks_cluster.this[0]
  +$73

    + EKS cluster
      +$73

+ module.eks.module.cluster.module.eks_managed_node_group["ng_ids-infracost-mock-44956be29f34"].aws_eks_node_group.this[0]
  +$32

    + module.eks.module.cluster.module.eks_managed_node_group["ng_ids-infracost-mock-44956be29f34"].aws_launch_template.this[0]
    
        + Instance usage (Linux/UNIX, on-demand, t3.medium)
          +$30
    
        + EC2 detailed monitoring
          +$2

+ module.eks.module.cluster.module.kms.aws_kms_key.this[0]
  +$1

    + Customer master key
      +$1

    + Requests
      Monthly cost depends on usage
        +$0.03 per 10k requests

    + ECC GenerateDataKeyPair requests
      Monthly cost depends on usage
        +$0.10 per 10k requests

    + RSA GenerateDataKeyPair requests
      Monthly cost depends on usage
        +$0.10 per 10k requests

- module.cluster.aws_cloudwatch_log_group.this[0]
  Monthly cost depends on usage

    - Data ingested
      Monthly cost depends on usage
        -$0.50 per GB

    - Archival Storage
      Monthly cost depends on usage
        -$0.03 per GB

    - Insights queries data scanned
      Monthly cost depends on usage
        -$0.005 per GB

+ module.eks.module.cluster.aws_cloudwatch_log_group.this[0]
  Monthly cost depends on usage

    + Data ingested
      Monthly cost depends on usage
        +$0.50 per GB

    + Archival Storage
      Monthly cost depends on usage
        +$0.03 per GB

    + Insights queries data scanned
      Monthly cost depends on usage
        +$0.005 per GB

- module.cluster.module.eks_managed_node_group["standard_spot"].aws_eks_node_group.this[0]
  -$32

    - module.cluster.module.eks_managed_node_group["standard_spot"].aws_launch_template.this[0]
    
        - Instance usage (Linux/UNIX, on-demand, t3.medium)
          -$30
    
        - EC2 detailed monitoring
          -$2

- module.cluster.aws_eks_cluster.this[0]
  -$73

    - EKS cluster
      -$73

Monthly cost change for binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks-demoapps/cluster (Module path: apps-devstg/us-east-1/k8s-eks-demoapps/cluster)
Amount:  +$1 ($105 → $106)
Percent: +1%

──────────────────────────────────
Project: apps-devstg-us-east-1-k8s-eks-demoapps-network
Module path: apps-devstg/us-east-1/k8s-eks-demoapps/network

+ module.vpc.module.vpc.aws_nat_gateway.this[0]
  +$33

    + NAT gateway
      +$33

    + Data processed
      Monthly cost depends on usage
        +$0.045 per GB

+ module.vpc.module.vpc_flow_logs[0].aws_s3_bucket.this
  Monthly cost depends on usage

    + Standard
    
        + Storage
          Monthly cost depends on usage
            +$0.023 per GB
    
        + PUT, COPY, POST, LIST requests
          Monthly cost depends on usage
            +$0.005 per 1k requests
    
        + GET, SELECT, and all other requests
          Monthly cost depends on usage
            +$0.0004 per 1k requests
    
        + Select data scanned
          Monthly cost depends on usage
            +$0.002 per GB
    
        + Select data returned
          Monthly cost depends on usage
            +$0.0007 per GB

    + Standard - infrequent access
    
        + Storage
          Monthly cost depends on usage
            +$0.0125 per GB
    
        + PUT, COPY, POST, LIST requests
          Monthly cost depends on usage
            +$0.01 per 1k requests
    
        + GET, SELECT, and all other requests
          Monthly cost depends on usage
            +$0.001 per 1k requests
    
        + Lifecycle transition
          Monthly cost depends on usage
            +$0.01 per 1k requests
    
        + Retrievals
          Monthly cost depends on usage
            +$0.01 per GB
    
        + Select data scanned
          Monthly cost depends on usage
            +$0.002 per GB
    
        + Select data returned
          Monthly cost depends on usage
            +$0.01 per GB

Monthly cost change for binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks-demoapps/network (Module path: apps-devstg/us-east-1/k8s-eks-demoapps/network)
Amount:  +$33 ($0.00 → $33)

──────────────────────────────────
Project: apps-devstg-us-east-1-k8s-eks-cluster
Module path: apps-devstg/us-east-1/k8s-eks/cluster

+ module.eks.module.cluster.aws_eks_cluster.this[0]
  +$73

    + EKS cluster
      +$73

+ module.eks.module.cluster.module.eks_managed_node_group["ng_ids-infracost-mock-44956be29f34"].aws_eks_node_group.this[0]
  +$32

    + module.eks.module.cluster.module.eks_managed_node_group["ng_ids-infracost-mock-44956be29f34"].aws_launch_template.this[0]
    
        + Instance usage (Linux/UNIX, on-demand, t3.medium)
          +$30
    
        + EC2 detailed monitoring
          +$2

+ module.eks.module.cluster.module.kms.aws_kms_key.this[0]
  +$1

    + Customer master key
      +$1

    + Requests
      Monthly cost depends on usage
        +$0.03 per 10k requests

    + ECC GenerateDataKeyPair requests
      Monthly cost depends on usage
        +$0.10 per 10k requests

    + RSA GenerateDataKeyPair requests
      Monthly cost depends on usage
        +$0.10 per 10k requests

- module.cluster.aws_cloudwatch_log_group.this[0]
  Monthly cost depends on usage

    - Data ingested
      Monthly cost depends on usage
        -$0.50 per GB

    - Archival Storage
      Monthly cost depends on usage
        -$0.03 per GB

    - Insights queries data scanned
      Monthly cost depends on usage
        -$0.005 per GB

+ module.eks.module.cluster.aws_cloudwatch_log_group.this[0]
  Monthly cost depends on usage

    + Data ingested
      Monthly cost depends on usage
        +$0.50 per GB

    + Archival Storage
      Monthly cost depends on usage
        +$0.03 per GB

    + Insights queries data scanned
      Monthly cost depends on usage
        +$0.005 per GB

- module.cluster.module.eks_managed_node_group["argocd"].aws_eks_node_group.this[0]
  -$32

    - module.cluster.module.eks_managed_node_group["argocd"].aws_launch_template.this[0]
    
        - Instance usage (Linux/UNIX, on-demand, t3.medium)
          -$30
    
        - EC2 detailed monitoring
          -$2

- module.cluster.module.eks_managed_node_group["monitoring"].aws_eks_node_group.this[0]
  -$32

    - module.cluster.module.eks_managed_node_group["monitoring"].aws_launch_template.this[0]
    
        - Instance usage (Linux/UNIX, on-demand, t3.medium)
          -$30
    
        - EC2 detailed monitoring
          -$2

- module.cluster.module.eks_managed_node_group["spot"].aws_eks_node_group.this[0]
  -$36

    - module.cluster.module.eks_managed_node_group["spot"].aws_launch_template.this[0]
    
        - Instance usage (Linux/UNIX, on-demand, t2.medium)
          -$34
    
        - EC2 detailed monitoring
          -$2

- module.cluster.aws_eks_cluster.this[0]
  -$438

    - EKS cluster (extended support)
      -$438

Monthly cost change for binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/cluster (Module path: apps-devstg/us-east-1/k8s-eks/cluster)
Amount:  -$432 ($539 → $106)
Percent: -80%

──────────────────────────────────
Project: apps-devstg-us-east-1-k8s-eks-network
Module path: apps-devstg/us-east-1/k8s-eks/network

+ module.vpc.module.vpc.aws_nat_gateway.this[0]
  +$33

    + NAT gateway
      +$33

    + Data processed
      Monthly cost depends on usage
        +$0.045 per GB

+ module.vpc.module.vpc_flow_logs[0].aws_s3_bucket.this
  Monthly cost depends on usage

    + Standard
    
        + Storage
          Monthly cost depends on usage
            +$0.023 per GB
    
        + PUT, COPY, POST, LIST requests
          Monthly cost depends on usage
            +$0.005 per 1k requests
    
        + GET, SELECT, and all other requests
          Monthly cost depends on usage
            +$0.0004 per 1k requests
    
        + Select data scanned
          Monthly cost depends on usage
            +$0.002 per GB
    
        + Select data returned
          Monthly cost depends on usage
            +$0.0007 per GB

    + Standard - infrequent access
    
        + Storage
          Monthly cost depends on usage
            +$0.0125 per GB
    
        + PUT, COPY, POST, LIST requests
          Monthly cost depends on usage
            +$0.01 per 1k requests
    
        + GET, SELECT, and all other requests
          Monthly cost depends on usage
            +$0.001 per 1k requests
    
        + Lifecycle transition
          Monthly cost depends on usage
            +$0.01 per 1k requests
    
        + Retrievals
          Monthly cost depends on usage
            +$0.01 per GB
    
        + Select data scanned
          Monthly cost depends on usage
            +$0.002 per GB
    
        + Select data returned
          Monthly cost depends on usage
            +$0.01 per GB

Monthly cost change for binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/network (Module path: apps-devstg/us-east-1/k8s-eks/network)
Amount:  +$33 ($0.00 → $33)

──────────────────────────────────
Key: * usage cost, ~ changed, + added, - removed
128 projects have no cost estimate changes.
Run the following command to see their breakdown: infracost breakdown --path=/path/to/code

──────────────────────────────────
*Usage costs can be estimated by updating Infracost Cloud settings, see docs for other options.

2633 cloud resources were detected:
∙ 596 were estimated
∙ 1929 were free
∙ 108 are not supported yet, see https://infracost.io/requested-resources:
  ∙ 37 x aws_identitystore_group_membership
  ∙ 33 x aws_identitystore_user
  ∙ 10 x aws_guardduty_member
  ∙ 7 x aws_identitystore_group
  ∙ 5 x aws_fms_policy
  ∙ 4 x aws_guardduty_detector
  ∙ 2 x aws_guardduty_organization_admin_account
  ∙ 2 x aws_guardduty_organization_configuration
  ∙ 2 x aws_organizations_delegated_administrator
  ∙ 1 x aws_cloudtrail_organization_delegated_admin_account
  ∙ 1 x aws_fms_admin_account
  ∙ 1 x aws_organizations_organization
  ∙ 1 x aws_route53_resolver_firewall_domain_list
  ∙ 1 x aws_route53_resolver_firewall_rule
  ∙ 1 x aws_route53_resolver_firewall_rule_group

Infracost estimate: Monthly estimate decreased by $366 ↓
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Changed project                                                  ┃ Baseline cost ┃ Usage cost* ┃ Total change ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ binbashar/le-tf-infra-aws/apps-...ast-1/k8s-eks-demoapps/cluster ┃           +$1 ┃           - ┃    +$1 (+1%) ┃
┃ binbashar/le-tf-infra-aws/apps-...ast-1/k8s-eks-demoapps/network ┃          +$33 ┃           - ┃         +$33 ┃
┃ binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/cluster  ┃         -$432 ┃           - ┃ -$432 (-80%) ┃
┃ binbashar/le-tf-infra-aws/apps-devstg/us-east-1/k8s-eks/network  ┃          +$33 ┃           - ┃         +$33 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

_{This comment will be updated when code changes.}

Added terraform modules for cluster and network, eks using them and keda

8c5ab87

juanmatias added feature patch labels Oct 31, 2024

juanmatias requested a review from a team as a code owner October 31, 2024 18:39

exequielrafaela assigned juanmatias Oct 31, 2024

juanmatias added 7 commits November 1, 2024 08:39

Removed old unused layers

4cbd414

Propagating terraform module changes to demoapps cluster

0b58e48

Fixing the cluster name suffix

cf6419e

Fixing way to call private zone id

1b74dd7

Fixing account names

fc4a40d

Propagating terraform module changes to base cluster

4d8e419

Added variable to set the CIDR used to assign IPs to PODs

83518d0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added terraform modules for cluster and network, eks using them and keda #648

Added terraform modules for cluster and network, eks using them and keda #648

juanmatias commented Oct 31, 2024 •

edited

Loading

github-actions bot commented Oct 31, 2024 •

edited

Loading

Added terraform modules for cluster and network, eks using them and keda #648

Are you sure you want to change the base?

Added terraform modules for cluster and network, eks using them and keda #648

Conversation

juanmatias commented Oct 31, 2024 • edited Loading

What?

Why?

References

github-actions bot commented Oct 31, 2024 • edited Loading

💰 Infracost report

Monthly estimate decreased by $366 📉

juanmatias commented Oct 31, 2024 •

edited

Loading

github-actions bot commented Oct 31, 2024 •

edited

Loading