Skip to content
This repository was archived by the owner on Jan 31, 2021. It is now read-only.

Initial Implementation #1

Merged
merged 43 commits into from
Jan 25, 2019
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
a8ebeb5
Add basic main.tf
joshmyers Jan 15, 2019
6990ce1
Add default global variables
joshmyers Jan 15, 2019
a1443ab
Add Postgres Aurora
joshmyers Jan 15, 2019
15dc1fc
Add VPC backing-services
joshmyers Jan 15, 2019
e16471d
Add Postgres read replica
joshmyers Jan 15, 2019
43d7c2e
Add Elasticache Redis backing service
joshmyers Jan 15, 2019
6fc9306
Add Kops metadata module
joshmyers Jan 15, 2019
f3381b3
Add AmazonMQ backing service for CodeFresh
joshmyers Jan 15, 2019
815a2af
Add docs
joshmyers Jan 15, 2019
8217a04
Remove RDS Aurora Postgres replica
joshmyers Jan 15, 2019
cc37e9d
Remove VPC and subnet modules
joshmyers Jan 15, 2019
703f468
Remove need for kops metadata
joshmyers Jan 16, 2019
8ea0f6f
Move AmazonMQ into own module
joshmyers Jan 16, 2019
90fd5c4
Add EFS
joshmyers Jan 16, 2019
e37c248
Drop Redis Elasticache version to 3.2.6
joshmyers Jan 16, 2019
93c40a7
Move aws_mq_broker users into module
joshmyers Jan 16, 2019
efbb2a9
Update docs
joshmyers Jan 16, 2019
07d5b12
Remove deprecated mq_broker_name variable
joshmyers Jan 18, 2019
f4d3da6
Pin aws-mq-broker module to 0.1.0 release
joshmyers Jan 18, 2019
4e7fca3
Add global enabled variable for whole module
joshmyers Jan 20, 2019
8878e8f
Add s3 bucket to CodeFresh backing services.
joshmyers Jan 20, 2019
ee895eb
Rename node_security_groups to security_groups
joshmyers Jan 20, 2019
5a83a29
Add usage to README
joshmyers Jan 20, 2019
f3c1acb
Pass only 1 or 2 subnets to mq.tf
joshmyers Jan 20, 2019
8d9805a
Actually use postgres_db_name if we pass it in
joshmyers Jan 20, 2019
f73c4e0
Add full example
joshmyers Jan 20, 2019
5af1915
Remove postgres_name variable
joshmyers Jan 21, 2019
4008913
Pin mq broker module to latest 0.2.0 release
joshmyers Jan 21, 2019
d5ff200
Remove redis_name as this is calculated in module
joshmyers Jan 21, 2019
4604abc
Update Redis variable descriptions
joshmyers Jan 21, 2019
4eb933c
overwrite SSM parameter is expected as a boolean
joshmyers Jan 21, 2019
2bb4d2a
Bump AmazonMQ default instance type
joshmyers Jan 21, 2019
7e06728
Remove null-label since not being used anymore
joshmyers Jan 21, 2019
f457c38
Bump aws-efs module
joshmyers Jan 23, 2019
87b1b80
Bump aws-s3-bucket to 0.1.0
joshmyers Jan 23, 2019
a842894
Remove aws-mq-broker enabled flags
joshmyers Jan 23, 2019
ab5df7e
Add optional EFS VPC and subnet_id variables
joshmyers Jan 23, 2019
862ef60
Fix typos
goruha Jan 24, 2019
84c69bc
Fix typos
goruha Jan 25, 2019
c66e25d
Remove EFS + AmazonMQ from CodeFresh services
joshmyers Jan 25, 2019
9bf608d
Remove Terraform glue variables
joshmyers Jan 25, 2019
62e276d
Update docs and pin example modules
joshmyers Jan 25, 2019
973fe02
Update docs to remove TODO and add note on enabled
joshmyers Jan 25, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
377 changes: 377 additions & 0 deletions README.md

Large diffs are not rendered by default.

24 changes: 17 additions & 7 deletions README.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ badges:
url: "https://slack.cloudposse.com"

related:
- name: "terraform-null-label"
description: "Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes])"
url: "https://github.com/cloudposse/terraform-null-label"
- name: "terraform-aws-rds-cluster"
description: "Terraform module to provision an RDS Aurora cluster for MySQL or Postgres"
url: "https://github.com/cloudposse/terraform-aws-rds-cluster"
Expand All @@ -56,22 +53,35 @@ related:

# Short description of this project
description: |-
Terraform module to provision CodeFresh Enterprise backing services for usage with a kops cluster
Terraform module to provision CodeFresh Enterprise backing services

introduction: |-
The module provisions the following resources:
- AWS Aurora primary
- AWS Aurora read replica
- AWS Aurora cluster
- AWS Elasticache Redis
- AWS AmazonMQ (ActiveMQ)
- AWS S3 bucket with associated user and permissions
- AWS EFS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- AWS EFS


# How to use this project
usage: |-
```terraform
module "codefresh_backing_services" {
source = "git::https://github.com/cloudposse/terraform-aws-codefresh-backing-services.git?ref=master"
enabled = "true"
name = "${var.name}"
namespace = "${var.namespace}"
region = "${var.region}"
stage = "${var.stage}"
vpc_id = "${module.vpc.vpc_id}"
subnet_ids = ["${module.subnets.private_subnet_ids}"]
security_groups = ["${module.vpc.vpc_default_security_group_id}"]
}
```

For a complete example, see [examples/complete](examples/complete)
TODO


include:
- "docs/targets.md"
- "docs/terraform.md"
Expand Down
181 changes: 181 additions & 0 deletions aurora-postgres.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Don't use `admin`
# Read more: <https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html>
# ("MasterUsername admin cannot be used as it is a reserved word used by the engine")
variable "postgres_admin_user" {
type = "string"
description = "Postgres admin user name"
default = ""
}

# Must be longer than 8 chars
# Read more: <https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html>
# ("The parameter MasterUserPassword is not a valid password because it is shorter than 8 characters")
variable "postgres_admin_password" {
type = "string"
description = "Postgres password for the admin user"
default = ""
}

variable "postgres_db_name" {
type = "string"
description = "Postgres database name"
default = ""
}

# db.r4.large is the smallest instance type supported by Aurora Postgres
# https://aws.amazon.com/rds/aurora/pricing
variable "postgres_instance_type" {
type = "string"
default = "db.r4.large"
description = "EC2 instance type for Postgres cluster"
}

variable "postgres_cluster_size" {
type = "string"
default = "2"
description = "Postgres cluster size"
}

variable "postgres_cluster_enabled" {
type = "string"
default = ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
default = ""
default = "true"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@goruha This is not needed and will fall back on enabled for the whole module if we don't override a specific service enabled flag like this. It was done like this so we could turn enable/disable individual services if we needed, rather than the entire module enable/disabled

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For terraform-root-modules I 💯 agree. For this module, I am on the fence.

description = "Set to false to prevent the module from creating any resources"
}

variable "postgres_cluster_family" {
type = "string"
default = "aurora-postgresql9.6"
description = "Postgres cluster DB family. Currently supported values are `aurora-postgresql9.6` and `aurora-postgresql10`"
}

variable "postgres_maintenance_window" {
type = "string"
default = "sun:03:00-sun:04:00"
description = "Weekly time range during which system maintenance can occur, in UTC"
}

locals {
postgres_cluster_enabled = "${var.postgres_cluster_enabled != "" ? var.postgres_cluster_enabled : var.enabled}"
postgres_admin_user = "${length(var.postgres_admin_user) > 0 ? var.postgres_admin_user : join("", random_string.postgres_admin_user.*.result)}"
postgres_admin_password = "${length(var.postgres_admin_password) > 0 ? var.postgres_admin_password : join("", random_string.postgres_admin_password.*.result)}"
postgres_db_name = "${var.postgres_db_name != "" ? var.postgres_db_name : join("", random_pet.postgres_db_name.*.id)}"
}

module "aurora_postgres" {
source = "git::https://github.com/cloudposse/terraform-aws-rds-cluster.git?ref=tags/0.10.0"
namespace = "${var.namespace}"
stage = "${var.stage}"
name = "${var.name}"
engine = "aurora-postgresql"
cluster_family = "${var.postgres_cluster_family}"
instance_type = "${var.postgres_instance_type}"
cluster_size = "${var.postgres_cluster_size}"
admin_user = "${local.postgres_admin_user}"
admin_password = "${local.postgres_admin_password}"
db_name = "${local.postgres_db_name}"
db_port = "5432"
maintenance_window = "${var.postgres_maintenance_window}"
vpc_id = "${var.vpc_id}"
subnets = ["${var.subnet_ids}"]
zone_id = "${local.zone_id}"
security_groups = ["${var.security_groups}"]
enabled = "${local.postgres_cluster_enabled}"
}

resource "random_pet" "postgres_db_name" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
separator = "_"
}

resource "random_string" "postgres_admin_user" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
length = 8
special = false
number = false
}

resource "random_string" "postgres_admin_password" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
length = 16
special = true
}

resource "aws_ssm_parameter" "aurora_postgres_database_name" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_database_name")}"
value = "${module.aurora_postgres.name}"
description = "Aurora Postgres Database Name"
type = "String"
overwrite = "${var.overwrite_ssm_parameter}"
}

resource "aws_ssm_parameter" "aurora_postgres_master_username" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_master_username")}"
value = "${module.aurora_postgres.user}"
description = "Aurora Postgres Username for the master DB user"
type = "String"
overwrite = "${var.overwrite_ssm_parameter}"
}

resource "aws_ssm_parameter" "aurora_postgres_master_password" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_master_password")}"
value = "${module.aurora_postgres.password}"
description = "Aurora Postgres Password for the master DB user"
type = "SecureString"
key_id = "${data.aws_kms_key.chamber_kms_key.id}"
overwrite = "${var.overwrite_ssm_parameter}"
}

resource "aws_ssm_parameter" "aurora_postgres_master_hostname" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_master_hostname")}"
value = "${module.aurora_postgres.master_host}"
description = "Aurora Postgres DB Master hostname"
type = "String"
overwrite = "${var.overwrite_ssm_parameter}"
}

resource "aws_ssm_parameter" "aurora_postgres_replicas_hostname" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_replicas_hostname")}"
value = "${module.aurora_postgres.replicas_host}"
description = "Aurora Postgres DB Replicas hostname"
type = "String"
overwrite = "${var.overwrite_ssm_parameter}"
}

resource "aws_ssm_parameter" "aurora_postgres_cluster_name" {
count = "${local.postgres_cluster_enabled == "true" ? 1 : 0}"
name = "${format(var.chamber_format, local.chamber_service, "aurora_postgres_cluster_name")}"
value = "${module.aurora_postgres.cluster_name}"
description = "Aurora Postgres DB Cluster Identifier"
type = "String"
overwrite = "${var.overwrite_ssm_parameter}"
}

output "aurora_postgres_database_name" {
value = "${local.postgres_cluster_enabled == "true" ? module.aurora_postgres.name : ""}"
description = "Aurora Postgres Database name"
}

output "aurora_postgres_master_username" {
value = "${local.postgres_cluster_enabled == "true" ? module.aurora_postgres.user : ""}"
description = "Aurora Postgres Username for the master DB user"
}

output "aurora_postgres_master_hostname" {
value = "${local.postgres_cluster_enabled == "true" ? module.aurora_postgres.master_host : ""}"
description = "Aurora Postgres DB Master hostname"
}

output "aurora_postgres_replicas_hostname" {
value = "${local.postgres_cluster_enabled == "true" ? module.aurora_postgres.replicas_host : ""}"
description = "Aurora Postgres Replicas hostname"
}

output "aurora_postgres_cluster_name" {
value = "${local.postgres_cluster_enabled == "true" ? module.aurora_postgres.cluster_name : ""}"
description = "Aurora Postgres Cluster Identifier"
}
10 changes: 10 additions & 0 deletions docs/targets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## Makefile Targets
```
Available targets:

help Help screen
help/all Display help for all targets
help/short This help short screen
lint Lint terraform code

```
98 changes: 98 additions & 0 deletions docs/terraform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| attributes | Additional attributes (e.g. `1`) | list | `<list>` | no |
| aws_assume_role_arn | | string | - | yes |
| chamber_format | Format to store parameters in SSM, for consumption with chamber | string | `/%s/%s` | no |
| chamber_service | `chamber` service name. See [chamber usage](https://github.com/segmentio/chamber#usage) for more details | string | `` | no |
| delimiter | Delimiter to be used between `namespace`, `stage`, `name` and `attributes` | string | `-` | no |
| efs_enabled | Set to false to prevent the module from creating any resources | string | `` | no |
| enabled | Set to false to prevent the module from creating any resources | string | `true` | no |
| kms_key_id | KMS key ID used to encrypt SSM parameters | string | `` | no |
| mq_apply_immediately | Specifies whether any cluster modifications are applied immediately, or during the next maintenance window | string | `true` | no |
| mq_audit_log | Enables audit logging. User management action made using JMX or the ActiveMQ Web Console is logged | string | `true` | no |
| mq_auto_minor_version_upgrade | Enables automatic upgrades to new minor versions for brokers, as Apache releases the versions | string | `false` | no |
| mq_deployment_mode | The deployment mode of the broker. Supported: SINGLE_INSTANCE and ACTIVE_STANDBY_MULTI_AZ | string | `ACTIVE_STANDBY_MULTI_AZ` | no |
| mq_enabled | Set to false to prevent the module from creating any resources | string | `` | no |
| mq_engine_type | The type of broker engine. Currently, Amazon MQ supports only ActiveMQ | string | `ActiveMQ` | no |
| mq_engine_version | The version of the broker engine. Currently, Amazon MQ supports only 5.15.0 or 5.15.6. | string | `5.15.0` | no |
| mq_general_log | Enables general logging via CloudWatch | string | `true` | no |
| mq_host_instance_type | The broker's instance type. e.g. mq.t2.micro or mq.m4.large | string | `mq.m4.large` | no |
| mq_maintenance_day_of_week | The day of the week. e.g. MONDAY, TUESDAY, or WEDNESDAY | string | `SUNDAY` | no |
| mq_maintenance_time_of_day | The time, in 24-hour format. e.g. 02:00 | string | `03:00` | no |
| mq_maintenance_time_zone | The time zone, in either the Country/City format, or the UTC offset format. e.g. CET | string | `UTC` | no |
| mq_publicly_accessible | Whether to enable connections from applications outside of the VPC that hosts the broker's subnets. | string | `false` | no |
| name | Name (e.g. `codefresh`) | string | `codefresh` | no |
| namespace | Namespace (e.g. `eg` or `cp`) | string | - | yes |
| overwrite_ssm_parameter | Whether to overwrite an existing SSM parameter | string | `true` | no |
| postgres_admin_password | Postgres password for the admin user | string | `` | no |
| postgres_admin_user | Postgres admin user name | string | `` | no |
| postgres_cluster_enabled | Set to false to prevent the module from creating any resources | string | `` | no |
| postgres_cluster_family | Postgres cluster DB family. Currently supported values are `aurora-postgresql9.6` and `aurora-postgresql10` | string | `aurora-postgresql9.6` | no |
| postgres_cluster_size | Postgres cluster size | string | `2` | no |
| postgres_db_name | Postgres database name | string | `` | no |
| postgres_instance_type | EC2 instance type for Postgres cluster | string | `db.r4.large` | no |
| postgres_maintenance_window | Weekly time range during which system maintenance can occur, in UTC | string | `sun:03:00-sun:04:00` | no |
| redis_apply_immediately | Whether to apply changes immediately or during the next maintenance window | string | `true` | no |
| redis_at_rest_encryption_enabled | Enable Redis encryption at rest | string | `true` | no |
| redis_auth_token | Auth token for password protecting Redis. `transit_encryption_enabled` must be set to `true`! Password must be longer than 16 chars | string | `` | no |
| redis_automatic_failover | Whether to enable automatic failover | string | `true` | no |
| redis_cluster_enabled | Set to false to prevent the module from creating any resources | string | `` | no |
| redis_cluster_size | Redis cluster size | string | `2` | no |
| redis_engine_version | Version of Redis engine | string | `3.2.6` | no |
| redis_instance_type | EC2 instance type for Redis cluster | string | `cache.t2.medium` | no |
| redis_maintenance_window | Weekly time range during which system maintenance can occur, in UTC | string | `sun:03:00-sun:04:00` | no |
| redis_params | A list of Redis parameters to apply. Note that parameters may differ from a Redis family to another | list | `<list>` | no |
| redis_transit_encryption_enabled | Enable TLS for Redis cluster | string | `true` | no |
| s3_access_key_name | S3 user IAM access key name for storing in SSM. Default to aws_acces_key_id so chamber exports as AWS_ACCESS_KEY_ID, a standard AWS IAM ENV variable | string | `aws_access_key_id` | no |
| s3_allowed_bucket_actions | List of actions to permit for S3 bucket | list | `<list>` | no |
| s3_enabled | Set to false to prevent the module from creating any resources | string | `` | no |
| s3_secret_key_name | S3 user IAM secret key name for storing in SSM. Default to aws_secret_acces_key so chamber exports as AWS_SECRET_ACCESS_KEY, a standard AWS IAM ENV variable | string | `aws_secret_access_key` | no |
| s3_user_enabled | Set to `true` to create an S3 user with permission to access the bucket | string | `` | no |
| s3_versioning_enabled | Whether to enable versioning on the S3 bucket. | string | `false` | no |
| security_groups | List of security groups to be allowed to connect to the CodeFresh backing services | list | `<list>` | no |
| stage | Stage (e.g. `prod`, `dev`, `staging`) | string | - | yes |
| subnet_ids | A list of subnet IDs to launch the CodeFresh backing services in | list | `<list>` | no |
| tags | Additional tags (e.g. map(`Cluster`,`us-east-1.cloudposse.co`) | map | `<map>` | no |
| vpc_id | VPC ID for the CodeFresh backing services | string | - | yes |
| zone_name | DNS zone name | string | - | yes |

## Outputs

| Name | Description |
|------|-------------|
| aurora_postgres_cluster_name | Aurora Postgres Cluster Identifier |
| aurora_postgres_database_name | Aurora Postgres Database name |
| aurora_postgres_master_hostname | Aurora Postgres DB Master hostname |
| aurora_postgres_master_username | Aurora Postgres Username for the master DB user |
| aurora_postgres_replicas_hostname | Aurora Postgres Replicas hostname |
| efs_arn | EFS arn |
| efs_dns_name | EFS DNS name |
| efs_host | EFS host |
| efs_id | EFS ID |
| efs_mount_target_dns_names | EFS mount target DNS names |
| efs_mount_target_ids | EFS mount target IDs |
| efs_mount_target_ips | EFS mount target IPs |
| efs_network_interface_ids | EFS network interface IDs |
| elasticache_redis_host | Elasticache Redis host |
| elasticache_redis_id | Elasticache Redis cluster ID |
| elasticache_redis_security_group_id | Elasticache Redis security group ID |
| mq_admin_username | AmazonMQ admin username |
| mq_application_username | AmazonMQ application username |
| mq_broker_arn | AmazonMQ broker ARN |
| mq_broker_id | AmazonMQ broker ID |
| mq_primary_ampq_ssl_endpoint | AmazonMQ primary AMQP+SSL endpoint |
| mq_primary_console_url | AmazonMQ active web console URL |
| mq_primary_ip_address | AmazonMQ primary IP address |
| mq_secondary_ampq_ssl_endpoint | AmazonMQ secondary AMQP+SSL endpoint |
| mq_secondary_console_url | AmazonMQ secondary web console URL |
| mq_secondary_ip_address | AmazonMQ secondary IP address |
| s3_access_key_id | The access key ID |
| s3_bucket_arn | The s3 bucket ARN |
| s3_secret_access_key | The secret access key. This will be written to the state file in plain-text |
| s3_user_arn | The ARN assigned by AWS for the user |
| s3_user_name | Normalized IAM user name |
| s3_user_unique_id | The user unique ID assigned by AWS |

Loading