Merge pull request #10018 from terraform-providers/f-keyvaluetags

internal/keyvaluetags: New Go package for consistently managing resource tags

GNUmakefile

@@ -9,6 +9,10 @@ default: build
 build: fmtcheck
 	go install
 
+gen:
+	rm -f aws/internal/keyvaluetags/*_gen.go
+	go generate ./...
+
 sweep:
 	@echo "WARNING: This will destroy infrastructure. Use only in development accounts."
 	go test $(TEST) -v -sweep=$(SWEEP) $(SWEEPARGS)
@@ -32,8 +36,8 @@ websitefmtcheck:
 
 lint:
 	@echo "==> Checking source code against linters..."
-	@golangci-lint run --no-config --deadline 5m --disable-all --enable staticcheck --exclude SA1019 --max-issues-per-linter 0 --max-same-issues 0 ./$(PKG_NAME)
-	@golangci-lint run ./$(PKG_NAME)
+	@golangci-lint run --no-config --deadline 5m --disable-all --enable staticcheck --exclude SA1019 --max-issues-per-linter 0 --max-same-issues 0 ./$(PKG_NAME)/...
+	@golangci-lint run ./$(PKG_NAME)/...
 	@tfproviderlint \
 		-c 1 \
 		-AT001 \
@@ -87,5 +91,5 @@ ifeq (,$(wildcard $(GOPATH)/src/$(WEBSITE_REPO)))
 endif
 	@$(MAKE) -C $(GOPATH)/src/$(WEBSITE_REPO) website-provider-test PROVIDER_PATH=$(shell pwd) PROVIDER_NAME=$(PKG_NAME)
 
-.PHONY: build sweep test testacc fmt fmtcheck lint tools test-compile website website-lint website-test
+.PHONY: build gen sweep test testacc fmt fmtcheck lint tools test-compile website website-lint website-test
 

aws/internal/keyvaluetags/README.md

@@ -0,0 +1,28 @@
+# keyvaluetags
+
+The `keyvaluetags` package is designed to provide a consistent interface for handling AWS resource key-value tags. Many of the AWS Go SDK services, implement their own Go struct with `Key` and `Value` fields (e.g. `athena.Tag`) while others simply use a map (e.g. `map[string]string`). These inconsistent implementations and numerous Go types makes the process of correctly working with each of the services a tedius, previously copy-paste-modify process.
+
+This package instead implements a single `KeyValueTags` type, which covers all key-value handling logic such as merging tags and ignoring keys via functions on the single type. The underlying implementation is compatible with Go operations such as `len()`.
+
+Full documentation for this package can be found on [GoDoc](https://godoc.org/github.com/terraform-providers/terraform-provider-aws/aws/internal/keyvaluetags).
+
+Many AWS Go SDK services that support tagging have their service-specific Go type conversion functions to and from `KeyValueTags` code generated. Converting from `KeyValueTags` to AWS Go SDK types is done via `{SERVICE}Tags()` functions on the type, while converting from AWS Go SDK types to the `KeyValueTags` type is done via `{SERVICE}KeyValueTags()` functions. For more information about this code generation, see the [`generators/servicetags` README](generators/servicetags/README.md).
+
+Some AWS Go SDK services that have common tag listing functionality (such as `ListTagsForResource` API call), also have auto-generated list functions. For more information about this code generation, see the [`generators/listtags` README](generators/listtags/README.md).
+
+Some AWS Go SDK services that have common tagging update functionality (such as `TagResource` and `UntagResource` API calls), also have auto-generated update functions. For more information about this code generation, see the [`generators/updatetags` README](generators/updatetags/README.md).
+
+## Code Structure
+
+```text
+aws/internal/keyvaluetags
+├── generators
+│   ├── listtags (generates list_tags_gen.go)
+│   ├── servicetags (generates service_tags_gen.go)
+│   └── updatetags (generates update_tags_gen.go)
+├── key_value_tags.go (core logic)
+├── list_tags_gen.go (generated AWS Go SDK service list tag functions)
+├── service_generation_customizations.go (shared AWS Go SDK service customizations for generators)
+├── service_tags_gen.go (generated AWS Go SDK service conversion functions)
+└── update_tags_gen.go (generated AWS Go SDK service tagging update functions)
+```

aws/internal/keyvaluetags/generators/listtags/README.md

@@ -0,0 +1,106 @@
+# listtags
+
+This package contains a code generator to consistently handle the various AWS Go SDK service implementations for listing resource tags. Not all AWS Go SDK services that support tagging are generated in this manner.
+
+To run this code generator, execute `go generate ./...` from the root of the repository. The general workflow for the generator is:
+
+- Generate Go file contents via template from local variables and functions
+- Go format file contents
+- Write file contents to `list_tags_gen.go` file
+
+## Example Output
+
+```go
+// AmplifyListTags lists amplify service tags.
+// The identifier is typically the Amazon Resource Name (ARN), although
+// it may also be a different identifier depending on the service.
+func AmplifyListTags(conn *amplify.Amplify, identifier string) (KeyValueTags, error) {
+    input := &amplify.ListTagsForResourceInput{
+        ResourceArn: aws.String(identifier),
+    }
+
+    output, err := conn.ListTagsForResource(input)
+
+    if err != nil {
+        return New(nil), err
+    }
+
+    return AmplifyKeyValueTags(output.Tags), nil
+}
+```
+
+## Implementing a New Generated Service
+
+### Requirements
+
+Before a new service can be added to the generator, the new service must:
+
+- Have the `KeyValueTags` conversion functions implemented for the AWS Go SDK service type/map. See also the [`servicetags` generator README](../servicetags/README.md).
+- Implement a function for listing resource tags (e.g. `ListTagsforResource`)
+- Have the service included in `aws/internal/keyvaluetags/service_generation_customizations.go`, if not present the following compilation error will be seen:
+
+```text
+2019/09/03 09:22:21 error executing template: template: listtags:19:41: executing "listtags" at <ClientType>: error calling ClientType: unrecognized ServiceClientType: acmpca
+```
+
+Once the service has met all the requirements, in `main.go`:
+
+- Add import for new service, e.g. `"github.com/aws/aws-sdk-go/service/athena"`
+- Add service name to `serviceNames`, e.g. `athena`
+- Run `go generate ./...` (or `make gen`) from the root of the repository to regenerate the code
+- Run `go test ./...` (or `make test`) from the root of the repository to ensure the generated code compiles
+- (Optional) Customize the service generation, if necessary (see below)
+
+### Customizations
+
+By default, the generator creates a `{SERVICE}ListTags()` function with the following structs and function calls:
+
+- `{SERVICE}.ListTagsForResourceInput` struct with `ResourceArn` field for calling `ListTagsForResource()` API call
+
+If these do not match the actual AWS Go SDK service implementation, the generated code will compile with errors. See the sections below for certain errors and how to handle them.
+
+#### ServiceListTagsFunction
+
+Given the following compilation error:
+
+```text
+./list_tags_gen.go:183:12: undefined: backup.ListTagsForResourceInput
+./list_tags_gen.go:187:21: conn.ListTagsForResource undefined (type *backup.Backup has no field or method ListTagsForResource)
+```
+
+The function for listing resource tags must be updated. Add an entry within the `ServiceListTagsFunction()` function of the generator to customize the naming of the `ListTagsForResource()` function and matching `ListTagsForResourceInput` struct. In the above case:
+
+```go
+case "backup":
+    return "ListTags"
+```
+
+#### ServiceListTagsInputIdentifierField
+
+Given the following compilation error:
+
+```text
+./list_tags_gen.go:1118:3: unknown field 'ResourceArn' in struct literal of type transfer.ListTagsForResourceInput
+```
+
+The field name to identify the resource for tag listing must be updated. Add an entry within the `ServiceListTagsInputIdentifierField()` function of the generator to customize the naming of the `ResourceArn` field for the list tags input struct. In the above case:
+
+```go
+case "transfer":
+    return "Arn"
+```
+
+#### ServiceListTagsOutputTagsField
+
+Given the following compilation error:
+
+```text
+./list_tags_gen.go:206:38: output.Tags undefined (type *cloudhsmv2.ListTagsOutput has no field or method Tags)
+```
+
+The field name of the tags from the tag listing must be updated. Add an entry within the `ServiceListTagsOutputTagsField()` function of the generator to customize the naming of the `Tags` field for the list tags output struct. In the above case:
+
+```go
+case "cloudhsmv2":
+    return "TagList"
+```