diff --git a/.github/contributing.md b/.github/contributing.md new file mode 100644 index 000000000..b7c27a5cc --- /dev/null +++ b/.github/contributing.md @@ -0,0 +1,34 @@ +# Contributing + +When contributing to this repository, please first discuss the change you wish to make via issue, +email, or any other method with the owners of this repository before making a change. + +Please note we have a code of conduct, please follow it in all your interactions with the project. + +## Pull Request Process + +1. Update the README.md with details of changes including example hcl blocks and [example files](./examples) if appropriate. +2. Run pre-commit hooks `pre-commit run -a`. +3. Once all outstanding comments and checklist items have been addressed, your contribution will be merged! Merged PRs will be included in the next release. The terraform-aws-vpc maintainers take care of updating the CHANGELOG as they merge. + +## Checklists for contributions + +- [ ] Add [semantics prefix](#semantic-pull-requests) to your PR or Commits (at least one of your commit groups) +- [ ] CI tests are passing +- [ ] README.md has been updated after any changes to variables and outputs. See https://github.com/terraform-aws-modules/terraform-aws-vpc/#doc-generation +- [ ] Run pre-commit hooks `pre-commit run -a` + +## Semantic Pull Requests + +To generate changelog, Pull Requests or Commits must have semantic and must follow conventional specs below: + +- `feat:` for new features +- `fix:` for bug fixes +- `improvement:` for enhancements +- `docs:` for documentation and examples +- `refactor:` for code refactoring +- `test:` for tests +- `ci:` for CI purpose +- `chore:` for chores stuff + +The `chore` prefix skipped during changelog generation. It can be used for `chore: update changelog` commit message by example. diff --git a/README.md b/README.md index 3977adf24..1cb32454e 100644 --- a/README.md +++ b/README.md @@ -120,6 +120,14 @@ You can add additional tags with `intra_subnet_tags` as with other subnet types. VPC Flow Log allows to capture IP traffic for a specific network interface (ENI), subnet, or entire VPC. This module supports enabling or disabling VPC Flow Logs for entire VPC. If you need to have VPC Flow Logs for subnet or ENI, you have to manage it outside of this module with [aws_flow_log resource](https://www.terraform.io/docs/providers/aws/r/flow_log.html). +### VPC Flow Log Examples + +By default `file_format` is `plain-text`. You can also specify `parquet` to have logs written in Apache Parquet format. + +``` +flow_log_file_format = "parquet" +``` + ### Permissions Boundary If your organization requires a permissions boundary to be attached to the VPC Flow Log role, make sure that you specify an ARN of the permissions boundary policy as `vpc_flow_log_permissions_boundary` argument. Read more about required [IAM policy for publishing flow logs](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html#flow-logs-iam). @@ -183,19 +191,25 @@ It is possible to integrate this VPC module with [terraform-aws-transit-gateway - [Manage Default VPC](https://github.com/terraform-aws-modules/terraform-aws-vpc/tree/master/examples/manage-default-vpc) - [Few tests and edge case examples](https://github.com/terraform-aws-modules/terraform-aws-vpc/tree/master/examples/issues) +## Contributing + +Report issues/questions/feature requests on in the [issues](https://github.com/terraform-aws-modules/terraform-aws-vpc/issues/new) section. + +Full contributing [guidelines are covered here](.github/contributing.md). + ## Requirements | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.38 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | >= 3.38 | +| [aws](#provider\_aws) | >= 3.63 | ## Modules @@ -370,8 +384,11 @@ No modules. | [flow\_log\_cloudwatch\_log\_group\_retention\_in\_days](#input\_flow\_log\_cloudwatch\_log\_group\_retention\_in\_days) | Specifies the number of days you want to retain log events in the specified log group for VPC flow logs. | `number` | `null` | no | | [flow\_log\_destination\_arn](#input\_flow\_log\_destination\_arn) | The ARN of the CloudWatch log group or S3 bucket where VPC Flow Logs will be pushed. If this ARN is a S3 bucket the appropriate permissions need to be set on that bucket's policy. When create\_flow\_log\_cloudwatch\_log\_group is set to false this argument must be provided. | `string` | `""` | no | | [flow\_log\_destination\_type](#input\_flow\_log\_destination\_type) | Type of flow log destination. Can be s3 or cloud-watch-logs. | `string` | `"cloud-watch-logs"` | no | +| [flow\_log\_file\_format](#input\_flow\_log\_file\_format) | (Optional) The format for the flow log. Valid values: `plain-text`, `parquet`. | `string` | `"plain-text"` | no | +| [flow\_log\_hive\_compatible\_partitions](#input\_flow\_log\_hive\_compatible\_partitions) | (Optional) Indicates whether to use Hive-compatible prefixes for flow logs stored in Amazon S3. | `bool` | `false` | no | | [flow\_log\_log\_format](#input\_flow\_log\_log\_format) | The fields to include in the flow log record, in the order in which they should appear. | `string` | `null` | no | | [flow\_log\_max\_aggregation\_interval](#input\_flow\_log\_max\_aggregation\_interval) | The maximum interval of time during which a flow of packets is captured and aggregated into a flow log record. Valid Values: `60` seconds or `600` seconds. | `number` | `600` | no | +| [flow\_log\_per\_hour\_partition](#input\_flow\_log\_per\_hour\_partition) | (Optional) Indicates whether to partition the flow log per hour. This reduces the cost and response time for queries. | `bool` | `false` | no | | [flow\_log\_traffic\_type](#input\_flow\_log\_traffic\_type) | The type of traffic to capture. Valid values: ACCEPT, REJECT, ALL. | `string` | `"ALL"` | no | | [igw\_tags](#input\_igw\_tags) | Additional tags for the internet gateway | `map(string)` | `{}` | no | | [instance\_tenancy](#input\_instance\_tenancy) | A tenancy option for instances launched into the VPC | `string` | `"default"` | no | diff --git a/examples/complete-vpc/README.md b/examples/complete-vpc/README.md index 4acc80200..83e222272 100644 --- a/examples/complete-vpc/README.md +++ b/examples/complete-vpc/README.md @@ -21,14 +21,14 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | >= 3.28 | +| [aws](#provider\_aws) | >= 3.63 | ## Modules diff --git a/examples/complete-vpc/versions.tf b/examples/complete-vpc/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/complete-vpc/versions.tf +++ b/examples/complete-vpc/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/ipv6/README.md b/examples/ipv6/README.md index b7ae18733..101c3a554 100644 --- a/examples/ipv6/README.md +++ b/examples/ipv6/README.md @@ -19,8 +19,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/ipv6/versions.tf b/examples/ipv6/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/ipv6/versions.tf +++ b/examples/ipv6/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/issues/README.md b/examples/issues/README.md index ff541a3ac..4a976e114 100644 --- a/examples/issues/README.md +++ b/examples/issues/README.md @@ -24,8 +24,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/issues/versions.tf b/examples/issues/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/issues/versions.tf +++ b/examples/issues/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/manage-default-vpc/README.md b/examples/manage-default-vpc/README.md index 2863e276e..a40caad59 100644 --- a/examples/manage-default-vpc/README.md +++ b/examples/manage-default-vpc/README.md @@ -21,8 +21,8 @@ Run `terraform destroy` when you don't need these resources. | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/manage-default-vpc/versions.tf b/examples/manage-default-vpc/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/manage-default-vpc/versions.tf +++ b/examples/manage-default-vpc/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/network-acls/README.md b/examples/network-acls/README.md index 26834dce4..b6cee0c30 100644 --- a/examples/network-acls/README.md +++ b/examples/network-acls/README.md @@ -23,8 +23,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/network-acls/versions.tf b/examples/network-acls/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/network-acls/versions.tf +++ b/examples/network-acls/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/outpost/README.md b/examples/outpost/README.md index 51ce3067a..be30e8211 100644 --- a/examples/outpost/README.md +++ b/examples/outpost/README.md @@ -23,14 +23,14 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | >= 3.28 | +| [aws](#provider\_aws) | >= 3.63 | ## Modules diff --git a/examples/outpost/versions.tf b/examples/outpost/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/outpost/versions.tf +++ b/examples/outpost/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/secondary-cidr-blocks/README.md b/examples/secondary-cidr-blocks/README.md index 29715df89..08764e25c 100644 --- a/examples/secondary-cidr-blocks/README.md +++ b/examples/secondary-cidr-blocks/README.md @@ -21,8 +21,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/secondary-cidr-blocks/versions.tf b/examples/secondary-cidr-blocks/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/secondary-cidr-blocks/versions.tf +++ b/examples/secondary-cidr-blocks/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/simple-vpc/README.md b/examples/simple-vpc/README.md index d5f6edde5..03494351c 100644 --- a/examples/simple-vpc/README.md +++ b/examples/simple-vpc/README.md @@ -25,8 +25,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/simple-vpc/versions.tf b/examples/simple-vpc/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/simple-vpc/versions.tf +++ b/examples/simple-vpc/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/examples/vpc-flow-logs/README.md b/examples/vpc-flow-logs/README.md index 10d782ae5..4dba2069b 100644 --- a/examples/vpc-flow-logs/README.md +++ b/examples/vpc-flow-logs/README.md @@ -23,15 +23,15 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | | [random](#requirement\_random) | >= 2 | ## Providers | Name | Version | |------|---------| -| [aws](#provider\_aws) | >= 3.28 | +| [aws](#provider\_aws) | >= 3.63 | | [random](#provider\_random) | >= 2 | ## Modules @@ -42,6 +42,7 @@ Note that this example may create resources which can cost money (AWS Elastic IP | [vpc\_with\_flow\_logs\_cloudwatch\_logs](#module\_vpc\_with\_flow\_logs\_cloudwatch\_logs) | ../../ | | | [vpc\_with\_flow\_logs\_cloudwatch\_logs\_default](#module\_vpc\_with\_flow\_logs\_cloudwatch\_logs\_default) | ../../ | | | [vpc\_with\_flow\_logs\_s3\_bucket](#module\_vpc\_with\_flow\_logs\_s3\_bucket) | ../../ | | +| [vpc\_with\_flow\_logs\_s3\_bucket\_parquet](#module\_vpc\_with\_flow\_logs\_s3\_bucket\_parquet) | ../../ | | ## Resources diff --git a/examples/vpc-flow-logs/main.tf b/examples/vpc-flow-logs/main.tf index 4dd51d3b2..61562358f 100644 --- a/examples/vpc-flow-logs/main.tf +++ b/examples/vpc-flow-logs/main.tf @@ -31,6 +31,25 @@ module "vpc_with_flow_logs_s3_bucket" { } } +module "vpc_with_flow_logs_s3_bucket_parquet" { + source = "../../" + + name = "vpc-flow-logs-s3-bucket" + cidr = "10.30.0.0/16" + + azs = ["${local.region}a"] + public_subnets = ["10.30.101.0/24"] + + enable_flow_log = true + flow_log_destination_type = "s3" + flow_log_destination_arn = module.s3_bucket.this_s3_bucket_arn + flow_log_file_format = "parquet" + + vpc_flow_log_tags = { + Name = "vpc-flow-logs-s3-bucket" + } +} + # CloudWatch Log Group and IAM role created automatically module "vpc_with_flow_logs_cloudwatch_logs_default" { source = "../../" diff --git a/examples/vpc-flow-logs/versions.tf b/examples/vpc-flow-logs/versions.tf index 2b46d6e06..8a45d6b8f 100644 --- a/examples/vpc-flow-logs/versions.tf +++ b/examples/vpc-flow-logs/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } random = { diff --git a/examples/vpc-separate-private-route-tables/README.md b/examples/vpc-separate-private-route-tables/README.md index 264e99fac..d25bcb04d 100644 --- a/examples/vpc-separate-private-route-tables/README.md +++ b/examples/vpc-separate-private-route-tables/README.md @@ -21,8 +21,8 @@ Note that this example may create resources which can cost money (AWS Elastic IP | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | -| [aws](#requirement\_aws) | >= 3.28 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | +| [aws](#requirement\_aws) | >= 3.63 | ## Providers diff --git a/examples/vpc-separate-private-route-tables/versions.tf b/examples/vpc-separate-private-route-tables/versions.tf index 7045f6d16..5a9fd0fc5 100644 --- a/examples/vpc-separate-private-route-tables/versions.tf +++ b/examples/vpc-separate-private-route-tables/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.28" + version = ">= 3.63" } } } diff --git a/modules/vpc-endpoints/README.md b/modules/vpc-endpoints/README.md index 13b15fbdd..e5d758b2b 100644 --- a/modules/vpc-endpoints/README.md +++ b/modules/vpc-endpoints/README.md @@ -55,7 +55,7 @@ module "endpoints" { | Name | Version | |------|---------| -| [terraform](#requirement\_terraform) | >= 0.12.31 | +| [terraform](#requirement\_terraform) | >= 0.13.1 | | [aws](#requirement\_aws) | >= 3.28 | ## Providers diff --git a/modules/vpc-endpoints/versions.tf b/modules/vpc-endpoints/versions.tf index 7045f6d16..ab4d354a9 100644 --- a/modules/vpc-endpoints/versions.tf +++ b/modules/vpc-endpoints/versions.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { diff --git a/variables.tf b/variables.tf index ddd592534..051743468 100644 --- a/variables.tf +++ b/variables.tf @@ -1151,3 +1151,26 @@ variable "outpost_az" { type = string default = null } + +variable "flow_log_file_format" { + description = "(Optional) The format for the flow log. Valid values: `plain-text`, `parquet`." + type = string + default = "plain-text" + validation { + condition = can(regex("^(plain-text|parquet)$", + var.flow_log_file_format)) + error_message = "ERROR valid values: plain-text, parquet." + } +} + +variable "flow_log_hive_compatible_partitions" { + description = "(Optional) Indicates whether to use Hive-compatible prefixes for flow logs stored in Amazon S3." + type = bool + default = false +} + +variable "flow_log_per_hour_partition" { + description = "(Optional) Indicates whether to partition the flow log per hour. This reduces the cost and response time for queries." + type = bool + default = false +} diff --git a/versions.tf b/versions.tf index 506304126..5a9fd0fc5 100644 --- a/versions.tf +++ b/versions.tf @@ -1,10 +1,10 @@ terraform { - required_version = ">= 0.12.31" + required_version = ">= 0.13.1" required_providers { aws = { source = "hashicorp/aws" - version = ">= 3.38" + version = ">= 3.63" } } } diff --git a/vpc-flow-logs.tf b/vpc-flow-logs.tf index c765be0fc..fea651f98 100644 --- a/vpc-flow-logs.tf +++ b/vpc-flow-logs.tf @@ -24,6 +24,12 @@ resource "aws_flow_log" "this" { vpc_id = local.vpc_id max_aggregation_interval = var.flow_log_max_aggregation_interval + destination_options { + file_format = var.flow_log_file_format + hive_compatible_partitions = var.flow_log_hive_compatible_partitions + per_hour_partition = var.flow_log_per_hour_partition + } + tags = merge(var.tags, var.vpc_flow_log_tags) }