Changelog Automation

.github/PULL_REQUEST_TEMPLATE.md

@@ -12,15 +12,6 @@
 <!--- If your PR fully resolves and should automatically close the linked issue, use Closes. Otherwise, use Relates --->
 Relates OR Closes #0000
 
-Release note for [CHANGELOG](https://github.com/hashicorp/terraform-provider-aws/blob/master/CHANGELOG.md):
-<!--
-If change is not user facing, just write "NONE" in the release-note block below.
--->
-
-```release-note
-
-```
-
 Output from acceptance testing:
 
 <!--

.github/workflows/generate_changelog.yml

@@ -0,0 +1,20 @@
+name: Generate CHANGELOG
+on:
+  pull_request:
+    types: [closed]
+jobs:
+  GenerateChangelog:
+    if: github.event.pull_request.merged
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - run: cd tools && go install github.com/hashicorp/go-changelog/cmd/changelog-build
+      - run: |
+          ./scripts/generate-changelog.sh
+          git config --local user.email "changelogbot@hashicorp.com"
+          git config --local user.name "changelogbot"
+          git add CHANGELOG.md
+          git commit -m "Update CHANGELOG.md for #${{ github.event.pull_request.number }}" 
+          git push   

.gitignore

@@ -27,6 +27,7 @@ website/node_modules
 *.iml
 log.txt
 markdown-link-check*.txt
+changelog.tmp
 
 # Test exclusions
 !command/test-fixtures/**/*.tfstate

GNUmakefile

@@ -50,6 +50,10 @@ gencheck:
 	@git diff --compact-summary --exit-code || \
 		(echo; echo "Unexpected difference in directories after code generation. Run 'make gen' command and commit."; exit 1)
 
+generate-changelog:
+	@echo "==> Generating changelog..."
+	@sh -c "'$(CURDIR)/scripts/generate-changelog.sh'"
+
 depscheck:
 	@echo "==> Checking source code with go mod tidy..."
 	@go mod tidy
@@ -115,6 +119,7 @@ tools:
 	cd tools && GO111MODULE=on go install github.com/katbyte/terrafmt
 	cd tools && GO111MODULE=on go install github.com/terraform-linters/tflint
 	cd tools && GO111MODULE=on go install github.com/pavius/impi/cmd/impi
+	cd tools && GO111MODULE=on go install github.com/hashicorp/go-changelog/cmd/changelog-build
 
 test-compile:
 	@if [ "$(TEST)" = "./..." ]; then \
@@ -149,4 +154,4 @@ website-lint-fix:
 	@docker run -v $(PWD):/markdown 06kellyjac/markdownlint-cli --fix website/docs/
 	@terrafmt fmt ./website --pattern '*.markdown'
 
-.PHONY: awsproviderlint build gen golangci-lint sweep test testacc fmt fmtcheck lint tools test-compile website-link-check website-lint website-lint-fix depscheck docscheck
+.PHONY: awsproviderlint build gen generate-changelog golangci-lint sweep test testacc fmt fmtcheck lint tools test-compile website-link-check website-lint website-lint-fix depscheck docscheck

docs/MAINTAINING.md

@@ -13,7 +13,6 @@
             - [tfproviderlint Updates](#tfproviderlint-updates)
             - [yaml.v2 Updates](#yaml-v2-updates)
     - [Pull Request Merge Process](#pull-request-merge-process)
-    - [Pull Request Types to CHANGELOG](#pull-request-types-to-changelog)
 - [Breaking Changes](#breaking-changes)
 - [Branch Dictionary](#branch-dictionary)
 - [Environment Variable Dictionary](#environment-variable-dictionary)
@@ -312,28 +311,6 @@ Run the acceptance testing pattern, `TestAccAWSCloudFormationStack(_dataSource)?
 The fix for this has been merged and will release with version X.Y.Z of the Terraform AWS Provider, expected in the XXX timeframe.
 ```
 
-### Pull Request Types to CHANGELOG
-
-The CHANGELOG is intended to show operator-impacting changes to the codebase for a particular version. If every change or commit to the code resulted in an entry, the CHANGELOG would become less useful for operators. The lists below are general guidelines on when a decision needs to be made to decide whether a change should have an entry.
-
-#### Changes that should have a CHANGELOG entry
-
-- New Resources and Data Sources
-- New full-length documentation guides (e.g. EKS Getting Started Guide, IAM Policy Documents with Terraform)
-- Resource and provider bug fixes
-- Resource and provider enhancements
-- Deprecations
-- Removals
-
-#### Changes that may have a CHANGELOG entry
-
-- Dependency updates: If the update contains relevant bug fixes or enhancements that affect operators, those should be called out.
-
-#### Changes that should _not_ have a CHANGELOG entry
-
-- Resource and provider documentation updates
-- Testing updates
-
 ## Breaking Changes
 
 When breaking changes to the provider are necessary we release them in a major version. If an issue or PR necessitates a breaking change, then the following procedure should be observed:

docs/contributing/pullrequest-submission-and-lifecycle.md

@@ -5,6 +5,7 @@
 - [Common Review Items](#common-review-items)
     - [Go Coding Style](#go-coding-style)
     - [Resource Contribution Guidelines](#resource-contribution-guidelines)
+- [Changelog Process](#changelog-process)
 
 We appreciate direct contributions to the provider codebase. Here's what to
 expect:
@@ -25,6 +26,8 @@ expect:
    or adding `[WIP]` to the beginning of the pull request title.
    Please include specific questions or items you'd like feedback on.
 
+1. Create a changelog entry following the process outlined [here](#changelog-process)
+
 1. Once you believe your pull request is ready to be reviewed, ensure the
    pull request is not a draft pull request by [marking it ready for review](https://help.github.com/en/articles/changing-the-stage-of-a-pull-request)
    or removing `[WIP]` from the pull request title if necessary, and a
@@ -39,8 +42,7 @@ expect:
 
 1. Once all outstanding comments and checklist items have been addressed, your
    contribution will be merged! Merged PRs will be included in the next
-   Terraform release. The provider team takes care of updating the CHANGELOG as
-   they merge.
+   Terraform release.
 
 1. In some cases, we might decide that a PR should be closed without merging.
    We'll make sure to provide clear reasoning when this happens.
@@ -184,3 +186,125 @@ The below are style-based items that _may_ be noted during review and are recomm
 - [ ] __Skips Timestamp Attributes__: Generally, creation and modification dates from the API should be omitted from the schema.
 - [ ] __Uses Paginated AWS Go SDK Functions When Iterating Over a Collection of Objects__: When the API for listing a collection of objects provides a paginated function, use it instead of looping until the next page token is not set. For example, with the EC2 API, [`DescribeInstancesPages`](https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#EC2.DescribeInstancesPages) should be used instead of [`DescribeInstances`](https://docs.aws.amazon.com/sdk-for-go/api/service/ec2/#EC2.DescribeInstances) when more than one result is expected.
 - [ ] __Adds Paginated Functions Missing from the AWS Go SDK to Internal Service Package__: If the AWS Go SDK does not define a paginated equivalent for a function to list a collection of objects, it should be added to a per-service internal package using the [`listpages` generator](../../aws/internal/generators/listpages/README.md). A support case should also be opened with AWS to have the paginated functions added to the AWS Go SDK.
+
+## Changelog Process
+
+HashiCorp’s open-source projects have always maintained user-friendly, readable CHANGELOGs that allow users to tell at a glance whether a release should have any effect on them, and to gauge the risk of an upgrade.
+
+We use the [go-changelog](https://github.com/hashicorp/go-changelog) to generate and update the changelog from files created in the `.changelog/` directory. It is important that when you raise your Pull Request, there is a changelog entry which describes the changes your contribution makes. Not all changes require an entry in the CHANGELOG, guidance follows on what changes do.
+
+### Changelog Format
+
+The changelog format requires an entry in the following format, where HEADER corresponds to the changelog category, and the entry is the changelog entry itself. The entry should be included in a file in the `.changelog` directory with the naming convention `{PR-NUMBER}.txt`. For example, to create a changelog entry for pull request 1234, there should be a file named `.changelog/1234.txt`.
+
+``````markdown
+```release-note:{HEADER}
+{ENTRY}
+```
+``````
+
+If a pull request should contain multiple changelog entries, then multiple blocks can be added to the same changelog file. For example:
+
+``````markdown
+```release-notes:note
+resource/aws_example_thing: The `broken` attribute has been deprecated. All configurations using `broken` should be updated to use the new `not_broken` attribute instead.
+```
+
+```release-notes:enhancement
+resource/aws_example_thing: Add `not_broken` attribute
+```
+``````
+
+### Pull Request Types to CHANGELOG
+
+The CHANGELOG is intended to show operator-impacting changes to the codebase for a particular version. If every change or commit to the code resulted in an entry, the CHANGELOG would become less useful for operators. The lists below are general guidelines and examples for when a decision needs to be made to decide whether a change should have an entry.
+
+#### Changes that should have a CHANGELOG entry
+
+##### New resource
+
+A new resource entry should only contain the name of the resource, and use the `release-note:new-resource` header.
+
+``````markdown
+```release-note:new-resource
+aws_secretsmanager_secret_policy
+```
+``````
+
+##### New data source
+
+A new datasource entry should only contain the name of the datasource, and use the `release-note:new-data-source` header.
+
+``````markdown
+```release-note:new-data-source
+aws_workspaces_workspace
+```
+``````
+
+##### New full-length documentation guides (e.g. EKS Getting Started Guide, IAM Policy Documents with Terraform)
+
+A new full length documentation entry gives the title of the documentation added, using the `release-note:new-guide` header.
+
+``````markdown
+```release-note:new-guide
+Custom Service Endpoint Configuration
+```
+``````
+
+##### Resource and provider bug fixes
+
+A new bug entry should use the `release-note:bug` header and have a prefix indicating the resource or datasource it corresponds to, a colon, then followed by a brief summary. Use a `provider` prefix for provider level fixes.
+
+``````markdown
+```release-note:bug
+resource/aws_glue_classifier: Fix quote_symbol being optional
+```
+``````
+
+##### Resource and provider enhancements
+
+A new enhancement entry should use the `release-note:enhancement` header and have a prefix indicating the resource or datasource it corresponds to, a colon, then followed by a brief summary. Use a `provider` prefix for provider level enchancements.
+
+``````markdown
+```release-note:enhancement
+resource/aws_eip: Add network_border_group argument
+```
+``````
+
+##### Deprecations
+
+A breaking-change entry should use the `release-note:note` header and have a prefix indicating the resource or datasource it corresponds to, a colon, then followed by a brief summary. Use a `provider` prefix for provider level changes.
+
+``````markdown
+```release-note:note
+resource/aws_dx_gateway_association: The vpn_gateway_id attribute is being deprecated in favor of the new associated_gateway_id attribute to support transit gateway associations
+```
+``````
+
+##### Breaking Changes and Removals
+
+A breaking-change entry should use the `release-note:breaking-change` header and have a prefix indicating the resource or datasource it corresponds to, a colon, then followed by a brief summary. Use a `provider` prefix for provider level changes.
+
+``````markdown
+```release-note:breaking-change
+resource/aws_lambda_alias: Resource import no longer converts Lambda Function name to ARN
+```
+``````
+
+#### Changes that may have a CHANGELOG entry
+
+Dependency updates: If the update contains relevant bug fixes or enhancements that affect operators, those should be called out.
+Any changes which do not fit into the above categories but warrant highlighting.
+Use resource/datasource/provider prefixes where appropriate.
+
+``````markdown
+```release-note:note
+resource/aws_lambda_alias: Resource import no longer converts Lambda Function name to ARN
+```
+``````
+
+#### Changes that should _not_ have a CHANGELOG entry
+
+- Resource and provider documentation updates
+- Testing updates
+- Code refactoring

scripts/changelog.tmpl

@@ -0,0 +1,41 @@
+
+{{- if index .NotesByType "breaking-change" }}
+BREAKING CHANGES:
+
+{{range index .NotesByType "breaking-change" -}}
+{{ template "note" .}}
+{{ end -}}
+{{- end -}}
+
+{{- if .NotesByType.note }}
+NOTES:
+
+{{range .NotesByType.note -}}
+{{ template "note" .}}
+{{ end -}}
+{{- end -}}
+
+{{- $features := combineTypes .NotesByType.feature (index .NotesByType "new-resource" ) (index .NotesByType "new-data-source") (index .NotesByType "new-guide") }}
+{{- if $features }}
+FEATURES:
+
+{{range $features | sort -}}
+{{ template "note" . }}
+{{ end -}}
+{{- end -}}
+
+{{- if .NotesByType.enhancement }}
+ENHANCEMENTS:
+
+{{range .NotesByType.enhancement | sort -}}
+{{ template "note" .}}
+{{ end -}}
+{{- end -}}
+
+{{- if .NotesByType.bug }}
+BUG FIXES:
+
+{{range .NotesByType.bug | sort -}}
+{{ template "note" . }}
+{{ end -}}
+{{- end -}}

scripts/generate-changelog.sh

@@ -0,0 +1,53 @@
+#!/bin/bash
+
+set -o errexit
+set -o nounset
+
+__dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+__parent="$(dirname "$__dir")"
+
+CHANGELOG_FILE_NAME="CHANGELOG.md"
+CHANGELOG_TMP_FILE_NAME="CHANGELOG.tmp"
+TARGET_SHA=$(git rev-parse HEAD)
+PREVIOUS_RELEASE_SHA=$(git rev-list -n 1 $(git describe --abbrev=0 --match='v*.*.*' --tags))
+
+if [ $TARGET_SHA == $PREVIOUS_RELEASE_SHA ]; then
+  echo "Nothing to do"
+  exit 0
+fi
+
+PREVIOUS_CHANGELOG=$(sed -n -e "/## $(git describe --abbrev=0 --match='v*.*.*' --tags | tr -d v)/,\$p" $__parent/$CHANGELOG_FILE_NAME)
+
+if [ -z "$PREVIOUS_CHANGELOG" ]
+then
+    echo "Unable to locate previous changelog contents."
+    exit 1
+fi 
+
+CHANGELOG=$($(go env GOPATH)/bin/changelog-build -this-release $TARGET_SHA \
+                      -last-release $PREVIOUS_RELEASE_SHA \
+                      -git-dir $__parent \
+                      -entries-dir .changelog \
+                      -changelog-template $__dir/changelog.tmpl \
+                      -note-template $__dir/release-note.tmpl)
+
+if [ -z "$CHANGELOG" ]
+then
+    echo "No changelog generated."
+    exit 0
+fi
+
+rm -f $CHANGELOG_TMP_FILE_NAME
+
+sed -n -e "1{/## /p;}" $__parent/$CHANGELOG_FILE_NAME > $CHANGELOG_TMP_FILE_NAME
+echo "$CHANGELOG" >> $CHANGELOG_TMP_FILE_NAME
+echo >> $CHANGELOG_TMP_FILE_NAME
+echo "$PREVIOUS_CHANGELOG" >> $CHANGELOG_TMP_FILE_NAME
+
+cp $CHANGELOG_TMP_FILE_NAME $CHANGELOG_FILE_NAME
+
+rm $CHANGELOG_TMP_FILE_NAME
+
+echo "Successfully generated changelog."
+
+exit 0

scripts/release-note.tmpl

@@ -0,0 +1,11 @@
+{{- define "note" -}}
+{{- if eq "new-resource" .Type -}}
+* **New Resource:** `{{.Body}}` (https://github.com/hashicorp/terraform-provider-aws/issues/{{- .Issue -}})
+{{- else if eq "new-data-source" .Type -}}
+* **New Data Source:** `{{.Body}}` (https://github.com/hashicorp/terraform-provider-aws/issues/{{- .Issue -}})
+{{- else if eq "new-guide" .Type -}}
+* **New Guide:** `{{.Body}}` (https://github.com/hashicorp/terraform-provider-aws/issues/{{- .Issue -}})
+{{- else -}}
+* {{.Body}} (https://github.com/hashicorp/terraform-provider-aws/issues/{{- .Issue -}})
+{{- end -}}
+{{- end -}}

tools/go.mod

@@ -6,6 +6,7 @@ require (
 	github.com/bflad/tfproviderdocs v0.8.0
 	github.com/client9/misspell v0.3.4
 	github.com/golangci/golangci-lint v1.35.2
+	github.com/hashicorp/go-changelog v0.0.0-20201005170154-56335215ce3a
 	github.com/katbyte/terrafmt v0.2.1-0.20200913185704-5ff4421407b4
 	github.com/terraform-linters/tflint v0.20.3
 )