Terraform EKS module

Description

Module provisions scalable EKS cluster with EC2 spot instances as worker nodes. By defaut terraform code will provision EKS cluster, IAM roles for worker nodes and cluster, additional IAM policies, security groups, instance profile, autoscaling groups and launch configurations for spot and on-demand worker nodes (autoscaling groups will be created per AZ for each launch configuration), ALB, Route53 record for ALB, target group. Also module will deploy cluster-autoscaler, spot termination handler, tiller, metric server and nginx-ingress-controller.

Deployment diagram:

Default EKS deployments:

Optionally following features can be enabled:

ACM certificate for ALB;
AWS WAF for whitelisting;
deploy fluend for container logs aggregation. Logs will be stored in AWS CloudWatch logs group:
deploy external-dns service:
deploy prometheus-operator with Grafana:

Requirements

Folloving resources shoud be created before cluster provisioning:

AWS VPC - https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html
AWS Route53 HostedZone

Tool	Version
terraform	==0.11.14
kubectl	>=1.13.4
helm	==2.13.1
aws-cli	==1.16.140
aws-iam-authenticator	==1.12.7

NOTES

rendered manifests and Helm charts for Kubernetes will be available in ${path.root}/manifests_rendered. Store it for further edition if required.

Usage

provider "aws" {
  region  = "us-east-1"
}

module "core" {
  source = "github.com/lean-delivery/tf-module-aws-core?ref=v0.2"

  project            = "eks"
  environment        = "test"
  availability_zones = ["us-east-1b","us-east-1c"]
  vpc_cidr           = "10.12.0.0/21"
  private_subnets    = ["10.12.0.0/24","10.12.1.0/24"]
  public_subnets     = ["10.12.2.0/24","10.12.3.0/24"]

  enable_nat_gateway = "true"
}

module "eks_test" {
  source = "github.com/lean-delivery/tf-module-aws-eks?ref=v1.0"

  project      = "eks"
  environment  = "test"

  cluster_version = "1.14"
  cluster_enabled_log_types = ["api"]

  vpc_id          = "${module.core.vpc_id}"
  private_subnets = "${module.core.private_subnets}"
  public_subnets  = "${module.core.public_subnets}"

  spot_configuration = [
      {
        instance_type = "m4.large",
        spot_price    = "0.05",
        asg_max_size  = "4",
        asg_min_size  = "0",
        asg_desired_capacity = "0",
        additional_kubelet_args = ""
      },
      {
        instance_type = "m4.xlarge",
        spot_price    = "0.08",
        asg_max_size  = "4",
        asg_min_size  = "1",
        asg_desired_capacity = "1",
        additional_kubelet_args = ""
      },
      {
        instance_type = "m4.2xlarge",
        spot_price    = "0.15",
        asg_max_size  = "4",
        asg_min_size  = "0",
        asg_desired_capacity = "0",
        additional_kubelet_args = ""
      }
    ]


  on_demand_configuration = [
      {
        instance_type = "m4.xlarge",
        asg_max_size  = "6",
        asg_min_size  = "0",
        asg_desired_capacity = "0",
        additional_kubelet_args = ""
      }
    ]

  service_on_demand_configuration = [
      {
        instance_type = "t3.small",
        asg_max_size  = "1",
        asg_min_size  = "1",
        asg_desired_capacity = "1",
        additional_kubelet_args = ""
      }
    ]

  worker_nodes_ssh_key      = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmYWeU1Hm+KfNmnOhB1OVh58KVcetUp6URTPB6fEOmIoNpXXpwFNeotjPoyFwwNc6KJ3LtDOo/Gx9SBkx9sSrHZcJVrKXRF/h4fe4nWeuoz0l3e8Toq+UajIXPjtv+mXkUX5LeyWKwInGc9U3BHXhzV8BYz9i1UqPDDvNsmep5gdRukI327Rh1G+kAYuhivvxbrzsIQrLUMjHqTiL25yILHZJ/eCJvcqLBXtxkPJThytVC1WUZ4vKQ5g8Ley6CtEa/7HolH6RlGduHswzqcdjrSMNxXPoSLF0j4cOeRy7MQA3TU4cLBgcmrwGgE5/IjBy3/3e15D3jtu8jX0r+tUR3 user@example.com"
  enable_waf                = true
  create_acm_certificate    = true
  root_domain               = "eks.example.com"
  alb_route53_record        = "eks-test.eks.example.com"
  alternative_domains       = ["*.eks.example.com"]
  alternative_domains_count = 1
  target_group_port         = "30081"
  cidr_whitelist = [
    {
      type  = "IPV4"
      value = "194.0.0.0/29"
    },
    {
      type  = "IPV4"
      value = "213.0.0.0/24"
    },
    ]

  deploy_ingress_controller       = true
  deploy_external_dns             = true
  enable_container_logs           = true
  container_logs_retention_days   = "5"
  enable_monitoring               = true
  monitoring_availability_zone    = "us-east-1c"
}

Inputs

Name	Description	Type	Default	Required
alb_ingress_rules	List of maps that contains ingress rules for ALB security group	list	`[{from_port = 80, to_port = 80, protocol = "tcp", cidr_blocks = "0.0.0.0/0"}, {from_port = 443, to_port = 443, protocol = "tcp", cidr_blocks = "0.0.0.0/0"}]`	no
alb_route53_record	Alias Route53 DNS record name for ALB	string	n/a	yes
alternative_domains	Alternative domains for ACM certificate dns records with ',' as delimiter	list	`[]`	no
alternative_domains_count	Alternative domains count for ACM certificate	string	`"0"`	no
service_on_demand_configuration	List of maps that contains configurations for ASGs with on-demand workers instances what will be used in EKS-cluster	list	`[{ instance_type = "t3.small", asg_max_size = "1", asg_min_size = "1", asg_desired_capacity = "1", additional_kubelet_args = ""}]`	no
cidr_whitelist	List of maps that contains IP CIDR with protocol type. Example provided in module examples	list	`[]`	no
cluster_enabled_log_types	A list of the desired control plane logging to enable. For more information, see Amazon EKS Control Plane Logging documentation	list	`[]`	no
cluster_version	Kubernetes version to use for the EKS cluster.	string	`"1.14"`	no
container_logs_retention_days	Set retention period for AWS CloudWatch log group with container logs	string	`"5"`	no
create_acm_certificate	Set true for ACM certificate for ALB creation	string	`"true"`	no
deploy_external_dns	Set true for External DNS installation.	string	`"false"`	no
deploy_ingress_controller	Set true for nginx ingress controller installation	string	`"true"`	no
enable_container_logs	Set true to install fluentd and store container logs in AWS CloudWatch log group	string	`"false"`	no
enable_monitoring	Set true for prometheus-operator and grafana deployment. Also storageClass will be created.	string	`"false"`	no
enable_waf	Set true to enable Web Application Firewall for whitelisting	string	`"false"`	no
environment	Environment name is used to identify resources	string	n/a	yes
local_exec_interpreter	Command to run for local-exec resources. Must be a shell-style interpreter. If you are on Windows Git Bash is a good choice.	list	`["/bin/sh", "-c"]`	no
map_accounts	Additional AWS account numbers to add to the aws-auth configmap. See variables.tf for example format.	list	`<list>`	no
map_accounts_count	The count of accounts in the map_accounts list.	string	`"0"`	no
map_roles	Additional IAM roles to add to the aws-auth configmap. See variables.tf for example format.	list	`<list>`	no
map_roles_count	The count of roles in the map_roles list.	string	`"0"`	no
map_users	Additional IAM users to add to the aws-auth configmap. See variables.tf for example format.	list	`<list>`	no
map_users_count	The count of roles in the map_users list.	string	`"0"`	no
monitoring_availability_zone	Availability zone in which will be deployed grafana and prometheus-operator, as this deployments required persistent volumes for data storing. If variable not set - availability zone of first subnet in private_subnets array will be used.	string	`""`	no
on_demand_configuration	List of maps that contains configurations for ASGs with on-demand workers instances what will be used in EKS-cluster	list	`[{instance_type = "m4.xlarge", asg_max_size = "6", asg_min_size = "0", asg_desired_capacity = "0", additional_kubelet_args = ""}]`	no
private_subnets	List of private subnets for cluster worker nodes provisioning	list	n/a	yes
project	Project name is used to identify resources	string	n/a	yes
public_subnets	List of public subnets for ALB provisioning	list	n/a	yes
root_domain	Root domain in which custom DNS record for ALB would be created	string	n/a	yes
spot_configuration	List of maps that contains configurations for ASGs with spot workers instances what will be used in EKS-cluster	list	`[{instance_type = "m4.large", spot_price = "0.05", asg_max_size = "4", asg_min_size = "1", asg_desired_capacity = "1", additional_kubelet_args = ""}, {instance_type = "m4.xlarge", spot_price = "0.08", asg_max_size = "4", asg_min_size = "0", asg_desired_capacity = "0", additional_kubelet_args = ""}]`	no
target_group_port	ALB targer group port. This value will be used as NodePort for Nginx Ingress controller service.	string	`"30080"`	no
volume_size	Volume size(GB) for worker node in cluster	string	`"50"`	no
vpc_id	VPC ID for cluster provisioning	string	n/a	yes
worker_nodes_ssh_key	If Public ssh key provided, will be used for ssh access to worker nodes. Otherwise instances will be created without ssh key.	string	`""`	no

Outputs

Name	Description
acm_certificate_arn	ACM certificate arn for ALB.
acm_certificate_domain	ACM certificate domain for ALB.
alb_arn	ALB arn for access to EKS deployments webUI.
alb_dns_name	ALB dns name for access to EKS deployments webUI.
alb_http_listener_arn	ALB http listener arn.
alb_https_listener_arn	ALB https listener arn.
alb_security_group_arn	ALB security group arn.
alb_security_group_id	ALB security group id.
alb_security_group_name	ALB security group name.
alb_target_group_arn	ALB target group arn.
alb_target_group_name	ALB target group name.
cluster_arn	The Amazon Resource Name (ARN) of the cluster.
cluster_certificate_authority_data	EKS cluster certificate.
cluster_endpoint	EKS cluster API endpoint.
cluster_iam_role_arn	IAM role ARN of the EKS cluster.
cluster_iam_role_name	IAM role name of the EKS cluster.
cluster_id	EKS cluster id.
cluster_security_group_id	EKS cluster security group id.
cluster_version	EKS cluster version.
config_map_aws_auth	A kubernetes configuration to authenticate to this EKS cluster.
iam_instance_profile_name	IAM instance profile name for EKS worker nodes.
kubeconfig	kubectl config file contents for this EKS cluster.
kubeconfig_filename	The filename of the generated kubectl config.
launch_configuration_on_demand_asg_names	Launch configuration name for EKS on-demand worker nodes.
launch_configuration_service_on_demand_asg_names	Launch configuration name for EKS non-scalable on-demand worker nodes.
launch_configuration_spot_asg_names	Launch configurations names for EKS spot worker nodes.
on_demand_asg_arns	EKS on-demand worker nodes ASGs arns.
on_demand_asg_ids	EKS on-demand worker nodes ASGs IDs.
on_demand_asg_names	EKS on-demand worker nodes ASGs names.
service_on_demand_asg_arns	EKS non-scalable on-demand worker nodes ASGs arns.
service_on_demand_asg_ids	EKS non-scalable on-demand worker nodes ASGs IDs.
service_on_demand_asg_names	EKS non-scalable on-demand worker nodes ASGs names.
path_to_manifests	Path to rendered manifests for EKS deployments.
spot_asg_arns	EKS spot worker nodes ASGs arns.
spot_asg_ids	EKS spot worker nodes ASGs IDs.
spot_asg_names	EKS spot worker nodes ASGs names.
ssh_key_name	SSH key name for worker nodes.
worker_iam_role_arn	IAM role ARN for EKS worker groups.
worker_iam_role_name	IAM role name for EKS worker groups.
worker_security_group_id	Security group ID attached to the EKS workers.

Terraform versions

Terraform version 0.11.14 or newer is required for this module to work.

Contributing

Thank you for your interest in contributing! Please refer to CONTRIBUTING.md for guidance.

License

Apache2.0 Licensed. See LICENSE for full details.

Authors

Lean Delivery Team team@lean-delivery.com

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
draw.io		draw.io
examples		examples
manifests		manifests
manifests_templates		manifests_templates
user_data		user_data
.gitignore		.gitignore
.travis.yml		.travis.yml
CONTRIBUTING.md		CONTRIBUTING.md
README.md		README.md
alb.tf		alb.tf
auto-scaling.tf		auto-scaling.tf
iam.tf		iam.tf
kubectl.tf		kubectl.tf
main.tf		main.tf
outputs.tf		outputs.tf
variables.tf		variables.tf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Terraform EKS module

Description

Deployment diagram:

Default EKS deployments:

Requirements

NOTES

Usage

Inputs

Outputs

Terraform versions

Contributing

License

Authors

About

Releases

Packages

Languages

alevkin/tf-module-aws-eks

Folders and files

Latest commit

History

Repository files navigation

Terraform EKS module

Description

Deployment diagram:

Default EKS deployments:

Requirements

NOTES

Usage

Inputs

Outputs

Terraform versions

Contributing

License

Authors

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages