Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure network ACLs for public/private/intra subnets #174

Closed
wants to merge 13 commits into from
Closed

Configure network ACLs for public/private/intra subnets #174

wants to merge 13 commits into from

Conversation

kinghuang
Copy link

Network ACLs provide a layer of security to VPCs with stateless rules for inbound and outbound network connections. This PR adds support for configuring network ACLs for public, private, and intra subnets using a series of variables.

Here is an example that implements some of the rules from Recommended Rules for Scenario 2.

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"
  version = "1.x.x"

  name = "acl-example"
  cidr = "10.64.0.0/16"

  azs              = ["us-east-2a", "us-east-2b", "us-east-2c"]
  private_subnets  = ["10.64.0.0/20", "10.64.16.0/20", "10.64.32.0/20"]
  public_subnets   = ["10.64.192.0/22", "10.64.196.0/22", "10.64.200.0/22"]

  public_inbound_acls = [
    "100", "allow", 80,    80,    "tcp", "0.0.0.0/0",       "Allows inbound HTTP traffic from any IPv4 address",
    "110", "allow", 443,   443,   "tcp", "0.0.0.0/0",       "Allows inbound HTTPS traffic from any IPv4 address",
    "120", "allow", 22,    22,    "tcp", "198.51.100.0/24", "Allows inbound SSH traffic from your home network",
    "130", "allow", 3389,  3389,  "tcp", "198.51.100.0/24", "Allows inbound RDP traffic from your home network",
    "140", "allow", 1024,  65535, "tcp", "0.0.0.0/0",       "Allows inbound return traffic from hosts"
  ]

  public_outbound_acls = [
    "100", "allow", 80,    80,    "tcp", "0.0.0.0/0",       "Allows outbound HTTP traffic from the subnet to the Internet",
    "110", "allow", 443,   443,   "tcp", "0.0.0.0/0",       "Allows outbound HTTPS traffic from the subnet to the Internet",
    "120", "allow", 1433,  1433,  "tcp", "10.64.0.0/17",    "Allows outbound MS SQL access to database servers in the private subnet",
    "140", "allow", 32768, 65535, "tcp", "0.0.0.0/0",       "Allows outbound responses to clients on the Internet",
    "150", "allow", 22,    22,    "tcp", "10.64.0.0/17",    "Allows outbound SSH access to instances in your private subnet"
  ]
}

aws_network_acl and aws_network_acl_rule resources are created for the ACL rules. An aws_default_network_acl resource is also created to adopt the default network ACL in the VPC.

Closes #173.

@kinghuang kinghuang changed the title WIP: Configure network ACLs for public/private/intra subnets Configure network ACLs for public/private/intra subnets Sep 26, 2018
@kinghuang
Copy link
Author

On further thought, maybe this is better done as a separate module. Otherwise, there's a lot of repetition involved, especially if the database, redshift, and elasticache subnets are covered, too.

@antonbabenko
Copy link
Member

Hi King, I like the code you have pushed so far very much.

I think it is important to provide all NACL rules in this module even if there will be some repetitions. The users of this module will benefit from having more features using one module.

Thank you for your contribution!

@kinghuang
Copy link
Author

I've added ACL resources to cover database, redshift, and elasticache subnets. And, refined the variable names to better reflect that they configure ACL rules, not ACLs.

@iveskins
Copy link

iveskins commented Nov 1, 2018

When I try to use this branch I get an error:
Error: module "vpc": "public_outbound_acls" is not a valid argument

Maybe this is because the variables are defined like "public_inbound_acl_rules", with '_rules' but the examples are written as "public_inbound_acls = ["
?

@iveskins
Copy link

I also got some strange errors, and couldn't refresh my state, after I inserted a new rule into the public_inbound_acls list
from:

 public_inbound_acls = [
    "900", "allow", 3389,  3389,  "tcp", "198.51.100.0/24", "Allows inbound RDP traffic from your home network",
    "901", "allow", 1024,  65535, "tcp", "0.0.0.0/0",       "Allows inbound return traffic from hosts"
  ]

to:

 public_inbound_acls = [
    "200", "allow", 22,    22,    "tcp", "198.51.100.0/24", "Allows inbound SSH traffic from your home network",
    "900", "allow", 3389,  3389,  "tcp", "198.51.100.0/24", "Allows inbound RDP traffic from your home network",
    "901", "allow", 1024,  65535, "tcp", "0.0.0.0/0",       "Allows inbound return traffic from hosts"
  ]

it seems like terraform was keeping track of the rules by their position the the list not the priority number.. so adding an item at the beginning caused problems

ws_route.private_nat_gateway[1]: Refreshing state... (ID: r-rtb-0c9743f91b1aa428c1080289494)

Error: Error refreshing state: 1 error(s) occurred:


* module.vpc.aws_network_acl_rule.public_inbound: 1 error(s) occurred:

* module.vpc.aws_network_acl_rule.public_inbound[4]: aws_network_acl_rule.public_inbound.4: Expected the Network ACL to have Entries, got: {
  Associations: [{
      NetworkAclAssociationId: "aclassoc-",
      NetworkAclId: "acl-",
      SubnetId: "subnet-"
    }],
  Entries: [
...

@dng-dev
Copy link

dng-dev commented Jan 18, 2019

When you dig into the code here is the line which creates the internal hash. So as you see they are using the rule number for this.

d.SetId(networkAclIdRuleNumberEgressHash(d.Get("network_acl_id").(string), d.Get("rule_number").(int), d.Get("egress").(bool), d.Get("protocol").(string)))

@tbugfinder
Copy link
Contributor

@kinghuang Will you finished on that pull request, looks like there are conflicts?

@ppieprzycki
Copy link

Any plans for merging this PR into master?

@antonbabenko
Copy link
Member

@ppieprzycki Yes, there are plans but I have not enough time in a day to cover all projects I work for free.

Regarding this PR, it requires some improvements related to conditional creation. I will fix and test it manually.

@kinghuang
Copy link
Author

Sorry, I forgot about the changes. I can rebase and take a stab at that today.

@jczerniak
Copy link

hi, why not use lookup function instead of element function?

With lookup function, our nacl definition is more readable and formatable.

  1. replace count
    from
    count = "${var.create_vpc && length(var.public_subnets) > 0 ? length(var.public_outbound_acl_rules) / 7 : 0}"
    to
    count = "${var.create_vpc && length(var.public_subnets) > 0 ? length(var.public_outbound_acl_rules) : 0}"

  2. replace element with lookup function
    from
    rule_number = "${element(var.public_inbound_acl_rules, count.index * 7 + 0)}"
    to
    rule_number = "${lookup(var.public_outbound_acl_rules[count.index], "rule_number")}"

  3. define nacl definitions with local variables

locals {
  network_acls_public = {
    public_inbound = [
      {
        rule_number = 100
        egress      = false
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 2222
        to_port     = 2222
      },
      {
        rule_number = 200
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 1024
        to_port     = 65535
      },
    ]
   public_outbound = [
      {
        rule_number = 100
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 23
        to_port     = 23
      },
      {
        rule_number = 200
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 1024
        to_port     = 65535
      },
    ]
  }
network_acls_default = {
    default_inbound = [
      {
        rule_number = 10
        egress      = false
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 2222
        to_port     = 2222
      },
      {
        rule_number = 20
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 1024
        to_port     = 65535
      },
    ]
    default_outbound = [
      {
        rule_number = 10
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 2222
        to_port     = 2222
      },
      {
        rule_number = 20
        egress      = true
        protocol    = "tcp"
        rule_action = "allow"
        cidr_block  = "0.0.0.0/0"
        from_port   = 1024
        to_port     = 65535
      },
    ]
  }
}
  1. invoke module with local variable
  • with defaults
    public_inbound_acl_rules = "${local.network_acls_default["default_inbound"]}"

  • custom with defaults
    public_inbound_acl_rules = "${concat(local.network_acls_default["default_inbound"],local.network_acls_public["public_inbound"]}"

I think that this will give more flexibility in a more complex environment

Regards
Jacek

@antonbabenko
Copy link
Member

I like the structure and ideas described by @jczerniak very much.

@kinghuang will you be able to update this PR?

Add variables for specifying network ACLs for public, private, and
intra subnets. The ACLs are defined in a list, with sets of seven
elements for the rule number, rule action, from port, to port,
protocol, and cidr block.
Add variables to specify additional tags for public, private, and intra
network ACL resources.
Add aws_network_acl and aws_network_acl_rule resources to specify
inbound and outbound network ACL rules for public, private, and intra
subnets.
Add a aws_default_network_acl resource to adopt the default network ACL
in the VPC.
Remove the empty lines after comment blocks for network ACLs to match
the style of the rest of this module.
Copy the simple-vpc example and adapt it to demonstrate the
configuration of network ACLs. A set of inbound and outbound ACLs are
specified in main.tf.
Clarify the variables for specifying ACL rules by renaming them from
*_acls to *_acl_rules. The values are used to create rules, not create
ACLs.
Add aws_network_acl and aws_network_acl_rule resources for database,
redshift, and elasticache subnets, along with corresponding variables.
This provides network ACL coverage to all subnet types produced by this
module.
For each subnet type, only create ACL resources if there are subnets
defined. For example, if database_subnets is empty, then don't create
ACL resources for database subnets.
Add the missing variable declarations for database_acl_tags,
redshift_acl_tags, and elasticache_acl_tags.
A single ACL is created for each of the subnet types. Update the
variable descriptions to reflect this.
Convert the NACL rule specifications from a list of lists to a list of
maps, as suggested by @jczerniak. This improves the readability of
rules.
Restructure the network ACL rules in the network-acls example to use
local variables to specify the rules, split between default and custom
rules.
@kinghuang
Copy link
Author

Following @jczerniak's suggestions, I've change the rules from lists of lists to lists of maps. And, updated the network-acls example to use locals and a concatenation of "default" and "custom" rules, instead of inline rules.

antonbabenko added a commit that referenced this pull request Mar 22, 2019
@antonbabenko antonbabenko mentioned this pull request Mar 22, 2019
5 tasks
antonbabenko added a commit that referenced this pull request Mar 22, 2019
* Add variables for network ACLs

Add variables for specifying network ACLs for public, private, and
intra subnets. The ACLs are defined in a list, with sets of seven
elements for the rule number, rule action, from port, to port,
protocol, and cidr block.

* Add variables for network ACL tags

Add variables to specify additional tags for public, private, and intra
network ACL resources.

* Add resources for network ACLs

Add aws_network_acl and aws_network_acl_rule resources to specify
inbound and outbound network ACL rules for public, private, and intra
subnets.

* Add resource for default network ACL

Add a aws_default_network_acl resource to adopt the default network ACL
in the VPC.

* Adjust spacing to match code style

Remove the empty lines after comment blocks for network ACLs to match
the style of the rest of this module.

* Copy simple-vpc example as network-acls

Copy the simple-vpc example and adapt it to demonstrate the
configuration of network ACLs. A set of inbound and outbound ACLs are
specified in main.tf.

* Rename variables from _acls to _acl_rules

Clarify the variables for specifying ACL rules by renaming them from
*_acls to *_acl_rules. The values are used to create rules, not create
ACLs.

* Add nacl resources and variables for other subnets

Add aws_network_acl and aws_network_acl_rule resources for database,
redshift, and elasticache subnets, along with corresponding variables.
This provides network ACL coverage to all subnet types produced by this
module.

* Create ACLs only if there are subnets

For each subnet type, only create ACL resources if there are subnets
defined. For example, if database_subnets is empty, then don't create
ACL resources for database subnets.

* Add missing variables for ACL tags

Add the missing variable declarations for database_acl_tags,
redshift_acl_tags, and elasticache_acl_tags.

* Make ACL singular in description for _acl_tags

A single ACL is created for each of the subnet types. Update the
variable descriptions to reflect this.

* Convert rules to nested list of maps

Convert the NACL rule specifications from a list of lists to a list of
maps, as suggested by @jczerniak. This improves the readability of
rules.

* Restructure example config to use locals

Restructure the network ACL rules in the network-acls example to use
local variables to specify the rules, split between default and custom
rules.

* Follow-up for #174
@antonbabenko
Copy link
Member

v1.60.0 has been released!

Huge thanks to @kinghuang for the great contribution!

guneriu added a commit to Yilu-Archive/terraform-aws-vpc that referenced this pull request May 14, 2021
* Fixing typo overriden -> overridden (terraform-aws-modules#150)

just a typo in the docs and in the
public_subnet_tags in the simple example

* Provide separate route tables for db/elasticache/redshift (terraform-aws-modules#155)

* Provide separate route tables for db/elasticache/redshift

* Added example for saperate routes

* Updated PR with suggestions

* Make redshift to use separate subnet route table also

* More cleanup and updates

* Fixed one more spelling mistake

* Add minimum support for IPv6 to VPC (terraform-aws-modules#156)

* Added support for IPv6 to VPC

* Removed IPv6 from outputs (fixed terraform-aws-modules#157) (terraform-aws-modules#158)

* Add secondary CIDR block support (terraform-aws-modules#163)

* Add secondary CIDR block support using a local variable to derive the vpc id to ensure the CIDR block operations are applied before the CIDR operations

* Add secondary cidr block outputs to module output

* Add the wonderful examples from matthiasr's PR located at terraform-aws-modules#162 all credit goes to them for this wonderful example

* From copy and paste accidentally used variable name that differed from these variables

* Resolve typo in secondary_cidr_blocks

* Fixed README formatting

* Followups for terraform-aws-modules#161

* Added local.vpc_id with description

* add vars for custom subnet and route table names (terraform-aws-modules#168)

* add vars for custom subnet and route table names

* revert db suffix to "db"

* Added cloudcraft.co as a sponsor for this module

* Added cloudcraft.co as a sponsor for this module

* Removed comments starting from # to fix README

* Updated link to cloudcraft

* Updated link to cloudcraft

* Reordering tag merging (terraform-aws-modules#148)

* Added amazon_side_asn to vpn_gateway (terraform-aws-modules#159)

* Added amazon_side_asn to vpn_gateway

* change to Amazon default ASN (as per API) (terraform-aws-modules#176)

https://docs.aws.amazon.com/cli/latest/reference/ec2/create-vpn-gateway.html

* Updated README.md after merge

* Fixed terraform-aws-modules#177 - public_subnets should not always be validated

* Fix for the error: module.vpc.aws_redshift_subnet_group.redshift: only lowercase alphanumeric characters and hyphens allowed in name

Read more: terraform-aws-modules#180

* Updated pre-commit version with new terraform-docs script

* Added IGW route for DB subnets (based on terraform-aws-modules#179)

* Reverted complete-example

* Added azs to outputs which is an argument

* Added possibility to control creation of elasticache and redshift subnet groups

* Added SSM and EC2 VPC endpoints (fixes terraform-aws-modules#195, terraform-aws-modules#194)

* adding option to create a route to nat gateway in database subnets

* Reordered vars in count for database_nat_gateway route

* add endpoints ec2messages, ssmmessages as those are required by Systems Manager in addition to ec2 and ssm.

* fix typo

* add additional endpoints to examples

* add files updated by pre-commit

* switch to terraform-docs v0.6.0

* Added option to create ECR api and dkr endpoints

* Added subnet ids to ecr endpoints

* Fixed formatting after terraform-aws-modules#205

* Fixed formatting after terraform-aws-modules#213

* Added intra subnet suffix. (terraform-aws-modules#220)

* Added intra subnet suffix.

* Fixed duplicate intra

* Fixed tag

* Added CHANGELOG.md (terraform-aws-modules#221)

* Bump version

* API gateway Endpoint (terraform-aws-modules#225)

* Updated changelog

* docs: Update comment in docs (terraform-aws-modules#226)

* Redshift public subnets (terraform-aws-modules#222)

* add public subnet for redshift to enable access for kinesis

* fix redshift subnet group name

* fix redshift public association

* add public redshift to documentation

* fix doc typo

* update code after review

* Redshift public subnets (terraform-aws-modules#222)

* Resolved conflicts after merge

* Updated changelog

* Network ACLs (terraform-aws-modules#238)

* Add variables for network ACLs

Add variables for specifying network ACLs for public, private, and
intra subnets. The ACLs are defined in a list, with sets of seven
elements for the rule number, rule action, from port, to port,
protocol, and cidr block.

* Add variables for network ACL tags

Add variables to specify additional tags for public, private, and intra
network ACL resources.

* Add resources for network ACLs

Add aws_network_acl and aws_network_acl_rule resources to specify
inbound and outbound network ACL rules for public, private, and intra
subnets.

* Add resource for default network ACL

Add a aws_default_network_acl resource to adopt the default network ACL
in the VPC.

* Adjust spacing to match code style

Remove the empty lines after comment blocks for network ACLs to match
the style of the rest of this module.

* Copy simple-vpc example as network-acls

Copy the simple-vpc example and adapt it to demonstrate the
configuration of network ACLs. A set of inbound and outbound ACLs are
specified in main.tf.

* Rename variables from _acls to _acl_rules

Clarify the variables for specifying ACL rules by renaming them from
*_acls to *_acl_rules. The values are used to create rules, not create
ACLs.

* Add nacl resources and variables for other subnets

Add aws_network_acl and aws_network_acl_rule resources for database,
redshift, and elasticache subnets, along with corresponding variables.
This provides network ACL coverage to all subnet types produced by this
module.

* Create ACLs only if there are subnets

For each subnet type, only create ACL resources if there are subnets
defined. For example, if database_subnets is empty, then don't create
ACL resources for database subnets.

* Add missing variables for ACL tags

Add the missing variable declarations for database_acl_tags,
redshift_acl_tags, and elasticache_acl_tags.

* Make ACL singular in description for _acl_tags

A single ACL is created for each of the subnet types. Update the
variable descriptions to reflect this.

* Convert rules to nested list of maps

Convert the NACL rule specifications from a list of lists to a list of
maps, as suggested by @jczerniak. This improves the readability of
rules.

* Restructure example config to use locals

Restructure the network ACL rules in the network-acls example to use
local variables to specify the rules, split between default and custom
rules.

* Follow-up for terraform-aws-modules#174

* Updated CHANGELOG

* Added missing VPC endpoints outputs (resolves terraform-aws-modules#246) (terraform-aws-modules#247)

* Updated CHANGELOG

* Add support for KMS VPC endpoint creation (terraform-aws-modules#243)

* Updated CHANGELOG

* Added ARN of VPC in module output (terraform-aws-modules#245)

I need in my policy generator the arn of vpc so I would like to include this

* Fixed formatting

* Updated CHANGELOG

* Add Output Of Subnet ARNs (terraform-aws-modules#242)

* Add Output Of Subnet ARNs

Facilitates resource access manager, subnet sharing across accounts

* Update Readme For Subnet ARN Output

* Fixed formatting

* Updated CHANGELOG

* Improving DHCP options docs (terraform-aws-modules#260)

* Improving DHCP options docs

* generating README from variables description

* Updated CHANGELOG

* ECS endpoint (terraform-aws-modules#261)

* add ecs vpc endpoints

* add ecs vpcendpoints outputs

* add ecs vpc endpoints to readme inputs/outputs table

* add ecs vpc endpoints to readme endpoint list

* Added VPC endpoints for SQS (closes terraform-aws-modules#248)

* Updated CHANGELOG

* Finally, Terraform 0.12 support (terraform-aws-modules#266)

* run terraform 0.12upgrade

* Cleanup for Terraform 0.12 (closes terraform-aws-modules#265, terraform-aws-modules#228)

* Fixed merge conflicts

* Updated CHANGELOG

* Upgrade Docker Image to fix CI (terraform-aws-modules#270)

* Added VPC Endpoints for SNS, Cloudtrail, ELB, Cloudwatch (terraform-aws-modules#269)

* Updated CHANGELOG

* Updated Terraform versions in README

* Updated CHANGELOG

* Fixed opportunity to create the vpc, vpn gateway routes (bug during upgrade to 0.12)

* Updated CHANGELOG

* Fixed broken 2.3.0

* Updated CHANGELOG

* Updated CHANGELOG

* Update tflint to 0.8.2 for circleci task (terraform-aws-modules#280)

* Updated VPC endpoint example (fixed terraform-aws-modules#249)

* Updated CHANGELOG

* Updated pre-commit-terraform to support terraform-docs and Terraform 0.12 (terraform-aws-modules#288)

* Updated CHANGELOG

* Enable backwards compatibility

* KAN-380 terraform 0.12 upgrade

* enable backwards compatibility

Co-authored-by: Tristan Escalada <tristan@escalada.us>
Co-authored-by: Anton Babenko <anton@antonbabenko.com>
Co-authored-by: Rupert Broad <rupert.broad@exact.com>
Co-authored-by: Scott Crooks <sc250024@users.noreply.github.com>
Co-authored-by: Mayur Nagekar <mayur@meetbeam.com>
Co-authored-by: ebarault <eric.barault@gmail.com>
Co-authored-by: tbugfinder <github@online.ms>
Co-authored-by: Michiel Dhadamus <michiel.dhadamus@dataminded.be>
Co-authored-by: Kinnaird McQuade <kmcquade@users.noreply.github.com>
Co-authored-by: tharun-allu <tharun-allu@users.noreply.github.com>
Co-authored-by: Kyle <1kylecameron@gmail.com>
Co-authored-by: bmihaescu <bmihaescu@gmail.com>
Co-authored-by: Nikos Loutas <nloutas@gmail.com>
Co-authored-by: Rafael Bernardo <rafaelbernardo@protonmail.com>
Co-authored-by: Blaine Schanfeldt <git@blaines.me>
Co-authored-by: Andreas Wittig <andreas@widdix.de>
Co-authored-by: Ilia Lazebnik <Ilia.lazebnik@gmail.com>
Co-authored-by: Niklas Wagner <Skaro@Skaronator.com>
Co-authored-by: Sebastian Geidies <sebastian.geidies@bcgdv.com>
Co-authored-by: ugur.guneri <ugur.guneri@yiluhub.com>
@github-actions
Copy link

github-actions bot commented Nov 4, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Configure network ACLs for subnets
7 participants