Skip to content

Another opnionated AWS VPC module, designed with GCC usage in mind

License

Notifications You must be signed in to change notification settings

GovTechSG/terraform-aws-vpc-ace

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

83 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

VPC

Opinionated module to create/provision a vpc, from GovTechSG/terraform-aws-vpc-forked

As this module is intended to use with a EKS cluster, additional tags to be included in subnets are required, this is to prevent situations where this module is reapplied after EKS has been created, which will remove the tags that are automatically added by EKS.

Also, it is modified to work with GCC VPC resources, where there are multiple permission boundaries applied and our terraform permissions are restricted from modifying and creating certain objects.

Gotchas with GCC

  1. you cannot modify anything that has tag Owner: GCCI

Solution: Do not ever tag any of your resources with this value You will notice that a number of resources in VPC dashboard that you cannot modify, not even adding additional tags. If you require additional tags, in the case of setting up EKS, where you need to tag the VPC https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html you will have to raise an SR to have someone from the technical desk to add it for you.

  1. you cannot create NAT Gateways

This module will fail as it tries to create NAT Gateway for you, which GCC does not. Therefore you will have to either, raise a SR to create them, or raise a SR to request for a temporary lift of the permission boundaries for you to apply this module (Recommended)

Upgrade

from v1 to v2

In v2.0 onwards, this module will no longer try to compute subnet cidrs using cidrsubnet functions and rely on user input to enter the cidr ranges for each subnet by themselves, see usage for example

Upgrade from v2 to v3

Splitting of VPC endpoints to submodule

The original VPC module has since moved away from creating individual VPC endpoints within the module and decoupled it from the main module. See PR

You will have to implement the submodule separately(import and apply), before updating to v3 on this module.

Setup your code as shown in the below example, after which, run terraform import 'aws_vpc_endpoint.this["kms"]' vpce-xxxx etc until you have imported all your endpoint services. You may now apply the vpc endpoint submodule, and you should not see any changes to your endpoints that will impact your existing applications, e.g changes to tags are okay, but changes to route table are not.

Examples
Terraform
module "endpoints" {
  source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"

  vpc_id             = "vpc-12345678"
  security_group_ids = ["sg-12345678"]

  endpoints = {
    s3 = {
      # interface endpoint
      service             = "s3"
      tags                = { Name = "s3-vpc-endpoint" }
    },
    dynamodb = {
      # gateway endpoint
      service         = "dynamodb"
      route_table_ids = ["rt-12322456", "rt-43433343", "rt-11223344"]
      tags            = { Name = "dynamodb-vpc-endpoint" }
    },
    sns = {
      service    = "sns"
      subnet_ids = ["subnet-12345678", "subnet-87654321"]
      tags       = { Name = "sns-vpc-endpoint" }
    },
    sqs = {
      service             = "sqs"
      private_dns_enabled = true
      security_group_ids  = ["sg-987654321"]
      subnet_ids          = ["subnet-12345678", "subnet-87654321"]
      tags                = { Name = "sqs-vpc-endpoint" }
    },
  }

  tags = {
    Owner       = "user"
    Environment = "dev"
  }
}
Terragrunt
dependency "vpc" {
  config_path = "../vpc"
}

inputs = {

  vpc_id             = dependency.vpc.outputs.vpc_id
  security_group_ids = ["sg-xxx"]
  subnet_ids         = dependency.vpc.outputs.private_subnets_ids

  endpoints = {
    s3 = {
      # gateway endpoint
      service      = "s3"
      service_type = "Gateway"
      tags         = { Name = "s3-vpc-endpoint" }
    },
    dynamodb = {
      # gateway endpoint
      service         = "dynamodb"
      service_type    = "Gateway"
      route_table_ids = concat(dependency.vpc.outputs.vpc_public_route_table_ids, dependency.vpc.outputs.vpc_private_route_table_ids, dependency.vpc.outputs.vpc_intra_route_table_ids)
      tags            = { Name = "dynamodb-vpc-endpoint" }
    },
    ec2 = {
      service             = "ec2"
      private_dns_enabled = true
      tags                = { Name = "ec2-vpc-endpoint" }
    }
    ecr = {
      service             = "ecr.api"
      private_dns_enabled = true
      tags                = { Name = "ecr-api-vpc-endpoint" }
    }
    elasticfilesystem = {
      service             = "elasticfilesystem"
      private_dns_enabled = true
      tags                = { Name = "elasticfilesystem-vpc-endpoint" }
    }
    kms = {
      service             = "kms"
      private_dns_enabled = true
      tags                = { Name = "kms-vpc-endpoint" }
    }
    execute-api = {
      service             = "execute-api"
      private_dns_enabled = true
      tags                = { Name = "execute-api-vpc-endpoint" }
    }
    sts = {
      service             = "sts"
      private_dns_enabled = true
      tags                = { Name = "sts-vpc-endpoint" }
    }
    "ecr.dkr" = {
      service             = "ecr.dkr"
      private_dns_enabled = true
      tags                = { Name = "ecr-dkr-vpc-endpoint" }
    }
    sqs = {
      service             = "sqs"
      private_dns_enabled = true
      tags                = { Name = "sqs-vpc-endpoint" }
    },
  }
}

Removing VPC endpoint state from VPC module

If you apply the VPC module immediately, you will notice that terraform will destroy your existing endpoints, this is not what we want. Since we have moved the state to be managed by the VPC endpoint submodule, we will need to remove the endpoint state from the VPC module.

For each of your endpoints, run terraform state rm "aws_vpc_endpoint.kms[0]"

Usage

There are 2 ways to use this module, mainly

  1. Create the vpc
  2. Reuse a created VPC(that is empty) and provision it. This is in the case of GCC where VPC will be created for us, and we have no rights to create new vpcs. In this scenario, use the terraform import command to import the resource to the module for it to manage. e.g terraform import 'module.vpc.aws_vpc.this[0]' vpc_xxxxxxx

Create vpc

module "vpc" {
  vpc_cidr = "172.1.1.0/25"
  secondary_cidr_blocks = ["172.2.2.0/24"]
  manage_cidr_block = "172.2.2.0/24"
  vpc_tags = {
    "DataClassification" = "Official Close"
    "Type" = "Internet"
  }

  public_subnets = [
    "172.2.2.0/27",
  ]
  database_subnets = [
    "172.2.2.32/27,
  ]

  private_subnets = [
    "172.2.2.64/27",

  ]
  number_of_azs = 2

}

Note the usage of secondary_cidr_blocks and manage_cidr_block As this module was originalyl intended to create 1 vpc with 1 cidr range for management, and it was only later discovered that GCC creates multiple cidr ranges in your VPC, you will have to use manage_cidr_block to tell the module to add create and manage resources for 1 cidr range at a time. Duplicate the module block for managing multiple cidrs as a workaround for now.

Reuse VPC

Terraform

terraform import 'module.vpc.aws_vpc.this[0]' vpc-xxxxxxxx terraform import 'module.vpc.aws_vpc_ipv4_cidr_block_association.this[0]' vpc-cidr-assoc-xxx terraform import 'module.vpc.aws_internet_gateway.this[0]' igw-xxx

Terragrunt

terragrunt import 'module.vpc.aws_vpc.this[0]' vpc-xxxxxxxx terragrunt import 'module.vpc.aws_vpc_ipv4_cidr_block_association.this[0]' vpc-cidr-assoc-xxx terragrunt import 'module.vpc.aws_internet_gateway.this[0]' igw-xxx

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

Name Source Version
vpc github.com/GovTechSG/terraform-aws-vpc-forked v4.0.0

Resources

Name Type
aws_default_network_acl.default resource
aws_default_security_group.default resource
aws_eip.nat resource
aws_network_acl.database resource
aws_network_acl.intra resource
aws_network_acl.private resource
aws_network_acl.public resource
aws_network_acl_rule.database resource
aws_network_acl_rule.database_inbound_allow_443_rule resource
aws_network_acl_rule.database_inbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.database_inbound_rdp_rule_deny resource
aws_network_acl_rule.database_inbound_ssh_rule_deny resource
aws_network_acl_rule.database_outbound_allow_443_rule resource
aws_network_acl_rule.database_outbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.database_outbound_ssh_rule_deny resource
aws_network_acl_rule.intra_inbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.intra_inbound_allow_all_udp resource
aws_network_acl_rule.intra_inbound_allow_all_udp_secondary_cidr resource
aws_network_acl_rule.intra_inbound_allow_tcp_dns resource
aws_network_acl_rule.intra_inbound_rdp_rule_deny resource
aws_network_acl_rule.intra_inbound_ssh_rule_deny resource
aws_network_acl_rule.intra_outbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.intra_outbound_allow_all_udp resource
aws_network_acl_rule.intra_outbound_allow_all_udp_secondary_cidr resource
aws_network_acl_rule.intra_outbound_allow_tcp_dns resource
aws_network_acl_rule.intra_outbound_rdp_rule_deny resource
aws_network_acl_rule.intra_outbound_ssh_rule_deny resource
aws_network_acl_rule.intranet_inbound_allow_443_rule resource
aws_network_acl_rule.intranet_inbound_bgp_179_rule resource
aws_network_acl_rule.intranet_inbound_bgp_179_rule_secondary_cidr resource
aws_network_acl_rule.intranet_inbound_nfs_111_rule resource
aws_network_acl_rule.intranet_inbound_nfs_111_rule_secondary_cidr resource
aws_network_acl_rule.intranet_inbound_ssh_rule resource
aws_network_acl_rule.intranet_inbound_ssh_rule_secondary_cidr resource
aws_network_acl_rule.intranet_outbound_allow_443_rule resource
aws_network_acl_rule.intranet_outbound_bgp_179_rule resource
aws_network_acl_rule.intranet_outbound_bgp_179_rule_secondary_cidr resource
aws_network_acl_rule.intranet_outbound_nfs_111_rule resource
aws_network_acl_rule.intranet_outbound_nfs_111_rule_secondary_cidr resource
aws_network_acl_rule.intranet_outbound_ssh_rule resource
aws_network_acl_rule.intranet_outbound_ssh_rule_secondary_cidr resource
aws_network_acl_rule.private_inbound_allow_443_rule resource
aws_network_acl_rule.private_inbound_allow_80_rule resource
aws_network_acl_rule.private_inbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.private_inbound_allow_all_udp resource
aws_network_acl_rule.private_inbound_allow_all_udp_secondary_cidr resource
aws_network_acl_rule.private_inbound_allow_bgp_179_rule resource
aws_network_acl_rule.private_inbound_allow_bgp_179_secondary_cidr resource
aws_network_acl_rule.private_inbound_allow_smtp_rule resource
aws_network_acl_rule.private_inbound_allow_tcp_dns resource
aws_network_acl_rule.private_inbound_ldap_rule resource
aws_network_acl_rule.private_inbound_ldap_rule_secondary_cidr resource
aws_network_acl_rule.private_inbound_nfs_111_rule resource
aws_network_acl_rule.private_inbound_nfs_111_rule_secondary_cidr resource
aws_network_acl_rule.private_inbound_openvpn_rule resource
aws_network_acl_rule.private_inbound_openvpn_rule_secondary_cidr resource
aws_network_acl_rule.private_inbound_rdp_rule_deny resource
aws_network_acl_rule.private_inbound_rdp_rule_deny_udp resource
aws_network_acl_rule.private_inbound_ssh_rule resource
aws_network_acl_rule.private_inbound_ssh_rule_deny resource
aws_network_acl_rule.private_inbound_ssh_rule_secondary_cidr resource
aws_network_acl_rule.private_outbound_allow_443_rule resource
aws_network_acl_rule.private_outbound_allow_80_rule resource
aws_network_acl_rule.private_outbound_allow_all_ephemeral_rule resource
aws_network_acl_rule.private_outbound_allow_all_udp resource
aws_network_acl_rule.private_outbound_allow_all_udp_secondary_cidr resource
aws_network_acl_rule.private_outbound_allow_bgp_179_rule resource
aws_network_acl_rule.private_outbound_allow_bgp_179_secondary_cidr resource
aws_network_acl_rule.private_outbound_allow_smtp_rule resource
aws_network_acl_rule.private_outbound_allow_tcp_dns resource
aws_network_acl_rule.private_outbound_ldap_rule resource
aws_network_acl_rule.private_outbound_ldap_rule_secondary_cidr resource
aws_network_acl_rule.private_outbound_nfs_111_rule resource
aws_network_acl_rule.private_outbound_nfs_111_rule_secondary_cidr resource
aws_network_acl_rule.private_outbound_openvpn_rule resource
aws_network_acl_rule.private_outbound_openvpn_rule_secondary_cidr resource
aws_network_acl_rule.private_outbound_rdp_rule_deny resource
aws_network_acl_rule.private_outbound_ssh_rule resource
aws_network_acl_rule.private_outbound_ssh_rule_deny resource
aws_network_acl_rule.private_outbound_ssh_rule_secondary_cidr resource
aws_network_acl_rule.public_inbound_allow_all_rule resource
aws_network_acl_rule.public_inbound_rdp_rule_deny resource
aws_network_acl_rule.public_inbound_rdp_rule_deny_udp resource
aws_network_acl_rule.public_inbound_ssh_rule resource
aws_network_acl_rule.public_inbound_ssh_rule_deny resource
aws_network_acl_rule.public_inbound_ssh_rule_secondary_cidr resource
aws_network_acl_rule.public_outbound_allow_all_rule resource
aws_network_acl_rule.public_outbound_rdp_rule_deny resource
aws_network_acl_rule.public_outbound_ssh_rule resource
aws_network_acl_rule.public_outbound_ssh_rule_deny resource
aws_network_acl_rule.public_outbound_ssh_rule_secondary_cidr resource
aws_security_group.allow_443 resource
aws_security_group.allow_http_https_outgoing resource
aws_availability_zones.available data source
aws_network_acls.default data source

Inputs

Name Description Type Default Required
aws_region Region to deploy current terraform script string "ap-southeast-1" no
cidr_name Name of cidr managed string "" no
create_flow_log_cloudwatch_iam_role Whether to create IAM role for VPC Flow Logs bool false no
create_flow_log_cloudwatch_log_group Whether to create CloudWatch log group for VPC Flow Logs bool false no
create_vpc Controls if VPC should be created (it affects almost all resources) bool true no
database_subnets cidr range of your database subnets list(string) [] no
default_network_acl_name Name to be used on the Default Network ACL string "" no
default_network_acl_tags Additional tags for the Default Network ACL map(string) {} no
default_route_table_name Name to be used on the default route table string null no
default_route_table_propagating_vgws List of virtual gateways for propagation list(string) [] no
default_route_table_routes Configuration block of routes. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/default_route_table#route list(map(string)) [] no
default_route_table_tags Additional tags for the default route table map(string) {} no
default_security_group_egress List of maps of egress rules to set on the default security group list(map(string)) [] no
default_security_group_ingress List of maps of ingress rules to set on the default security group list(map(string)) [] no
default_security_group_name Name to be used on the default security group string null no
default_security_group_rules Allowed inbound rules for default security group map(any) {} no
default_security_group_tags Additional tags for the default security group map(string) {} no
default_vpc_enable_dns_hostnames Should be true to enable DNS hostnames in the Default VPC bool true no
default_vpc_enable_dns_support Should be true to enable DNS support in the Default VPC bool true no
default_vpc_name Name to be used on the Default VPC string null no
default_vpc_tags Additional tags for the Default VPC map(string) {} no
eks_cluster_tags List of tags that EKS will create, but also added to VPC for persistency across terraform applies map(any) {} no
enable_flow_log Whether or not to enable VPC Flow Logs bool false no
enable_nat_gateway Should be true if you want to provision NAT Gateways for each of your private networks bool true no
firewall_dedicated_network_acl Whether to use dedicated network ACL (not default) and custom rules for firewall subnets bool false no
firewall_inbound_acl_rules firewall subnets inbound network ACL rules list(map(string))
[
{
"cidr_block": "0.0.0.0/0",
"from_port": 0,
"protocol": "-1",
"rule_action": "allow",
"rule_number": 100,
"to_port": 0
}
]
no
firewall_outbound_acl_rules Firewall subnets outbound network ACL rules list(map(string))
[
{
"cidr_block": "0.0.0.0/0",
"from_port": 0,
"protocol": "-1",
"rule_action": "allow",
"rule_number": 100,
"to_port": 0
}
]
no
firewall_subnets cidr range of your firewall subnets list(string) [] no
firewall_sync_states Output of aws_networkfirewall_firewall.firewall_status[0].sync_states
list(object({
attachment = list(object({
endpoint_id = string
subnet_id = string
}))
availability_zone = string
}))
[] no
flow_log_cloudwatch_iam_role_arn The ARN for the IAM role that's used to post flow logs to a CloudWatch Logs log group. When flow_log_destination_arn is set to ARN of Cloudwatch Logs, this argument needs to be provided string "" no
flow_log_cloudwatch_log_group_kms_key_id The ARN of the KMS Key to use when encrypting log data for VPC flow logs string null no
flow_log_cloudwatch_log_group_name_prefix Specifies the name prefix of CloudWatch Log Group for VPC flow logs string "/aws/vpc-flow-log/" no
flow_log_cloudwatch_log_group_name_suffix Specifies the name suffix of CloudWatch Log Group for VPC flow logs string "" no
flow_log_cloudwatch_log_group_retention_in_days Specifies the number of days you want to retain log events in the specified log group for VPC flow logs number null no
flow_log_destination_arn The ARN of the CloudWatch log group or S3 bucket where VPC Flow Logs will be pushed. If this ARN is a S3 bucket the appropriate permissions need to be set on that bucket's policy. When create_flow_log_cloudwatch_log_group is set to false this argument must be provided string "" no
flow_log_destination_type Type of flow log destination. Can be s3 or cloud-watch-logs string "cloud-watch-logs" no
flow_log_file_format (Optional) The format for the flow log. Valid values: plain-text, parquet string null no
flow_log_hive_compatible_partitions (Optional) Indicates whether to use Hive-compatible prefixes for flow logs stored in Amazon S3 bool false no
flow_log_log_format The fields to include in the flow log record, in the order in which they should appear string null no
flow_log_max_aggregation_interval The maximum interval of time during which a flow of packets is captured and aggregated into a flow log record. Valid Values: 60 seconds or 600 seconds number 600 no
flow_log_per_hour_partition (Optional) Indicates whether to partition the flow log per hour. This reduces the cost and response time for queries bool false no
flow_log_traffic_type The type of traffic to capture. Valid values: ACCEPT, REJECT, ALL string "ALL" no
folder Path relative to root of terraform directory where this module is used. This is for easier locating of where the individual resource is created with aws console map(any) n/a yes
intranet_subnets cidr range of your intranet subnets list(string) [] no
manage_default_network_acl Should be true to adopt and manage Default Network ACL bool false no
manage_default_route_table Should be true to manage default route table bool true no
manage_default_security_group Should be true to adopt and manage default security group bool true no
manage_default_vpc Should be true to adopt and manage Default VPC bool false no
map_public_ip_on_launch Should be false if you do not want to auto-assign public IP on launch bool true no
number_of_azs Determines number of availability zones to use in the region number 2 no
private_subnets cidr range of your private subnets list(string) [] no
public_subnets cidr range of your public subnets list(string) [] no
secondary_cidr_blocks List of secondary CIDR blocks to associate with the VPC to extend the IP Address pool list(string) [] no
tags Tags to apply to resources map(any) {} no
vpc_cidr CIDR for the VPC, check that this doesn't collide with an existing one string n/a yes
vpc_flow_log_permissions_boundary The ARN of the Permissions Boundary for the VPC Flow Log IAM Role string null no
vpc_flow_log_tags Additional tags for the VPC Flow Logs map(string) {} no
vpc_id VPC id for use in cases where VPC was already created and you would like to reuse it with this module. Not required if create_vpc = true string "" no
vpc_name Name of VPC string n/a yes
vpc_tags Tags to apply to VPC map(any) {} no

Outputs

Name Description
database_network_acl_id The ID of the database network ACL
database_subnet_group Group name of the database subnet
database_subnets_cidr_blocks CIDR blocks for database subnets for the VPC
database_subnets_ids Intranet subnets for the VPC
default_network_acl_id The ID of the default network ACL
default_security_group_id The ID of the security group created by default on VPC creation
firewall_network_acl_id The ID of the database network ACL
firewall_route_table_ids List of IDs of firewall route tables
firewall_subnet_arns List of ARNs of firewall subnets
firewall_subnets_cidr_blocks CIDR blocks for firewall subnets for the VPC
firewall_subnets_ids firewall subnets for the VPC
https_security_group_id The ID of the security group to all traffic from 443
intra_subnets_cidr_blocks CIDR blocks for intranet subnets for the VPC
intra_subnets_ids Intranet subnets for the VPC
intranet_network_acl_id The ID of the intra network ACL
private_network_acl_id The ID of the privatenetwork ACL
private_subnets_cidr_blocks CIDR blocks fo private subnets for the VPC
private_subnets_ids Private subnets for the VPC
public_network_acl_id The ID of the public network ACL
public_subnets_cidr_blocks CIDR blocks for public subnets for the VPC
public_subnets_ids Public subnets for the VPC
vpc_azs AZs at time of creation
vpc_cidr_block The CIDR block of the VPC
vpc_database_route_table_ids List of IDs of database route tables
vpc_id ID of the created VPC
vpc_igw_id IGW ID
vpc_intra_route_table_ids List of IDs of intra route tables
vpc_main_route_table_id The ID of the main route table associated with this VPC
vpc_nat_eip_ids EIP for the NAT gateway in the VPC
vpc_nat_eip_public Public address for the EIP on the NAT Gateway
vpc_nat_ids NAT gateway IDs
vpc_private_route_table_ids List of IDs of private route tables
vpc_public_route_table_ids The IDs of the public route tables
vpc_region The region the VPC belongs to