r/aws_dynamodb_table_gsi

[danquack]

Co-Authored-By: @ben-bourdin451. updated to v2 sdks from #6688

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates or Closes #17096
Relates or Closes #671

Output from acceptance testing:

$ TF_ACC=1 go test ././internal/service/dynamodb/... -v -count 1 -parallel 3  -run=TestAccAWSDynamoDbTableGsi_ -timeout 180m
=== RUN   TestAccAWSDynamoDbTableGsi_basic
=== PAUSE TestAccAWSDynamoDbTableGsi_basic
=== RUN   TestAccAWSDynamoDbTableGsi_updateCapacity
=== PAUSE TestAccAWSDynamoDbTableGsi_updateCapacity
=== RUN   TestAccAWSDynamoDbTableGsi_updateNonAttributes
=== PAUSE TestAccAWSDynamoDbTableGsi_updateNonAttributes
=== RUN   TestAccAWSDynamoDbTableGsi_delete
=== PAUSE TestAccAWSDynamoDbTableGsi_delete
=== CONT  TestAccAWSDynamoDbTableGsi_basic
=== CONT  TestAccAWSDynamoDbTableGsi_updateNonAttributes
=== CONT  TestAccAWSDynamoDbTableGsi_updateCapacity
--- PASS: TestAccAWSDynamoDbTableGsi_basic (280.10s)
=== CONT  TestAccAWSDynamoDbTableGsi_delete
--- PASS: TestAccAWSDynamoDbTableGsi_updateCapacity (309.75s)
--- PASS: TestAccAWSDynamoDbTableGsi_updateNonAttributes (691.06s)
--- PASS: TestAccAWSDynamoDbTableGsi_delete (431.54s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/dynamodb   714.664s

Contributor Notes:
Open for feedback...

• similar to aws security groups ingress/egress + security group rules, separating GSIs has an issue where attributes can be defined in different places. To avoid computed value differences when reading, I used a CustomizeDiff, but it clears the whole attribute. Not sure if you can clear just the attribute in the set in question.
• Along the same lines, i added a feature switch to manage it externally. Is this an acceptable approach?

[danquack]

@ewbankkit no need to review now. This is a WIP PR and will need some love based on feedback from the GitHub issues associated

[danquack]

Open for discussion, should we set the Deprecated bit?

[danquack]

@ewbankkit wondering if I could bump this for your review?

[ewbankkit]

@danquack Thanks for the work here 🎉 👏.

Our concern with attempting to decompose the DynamoDB Table resource into multiple smaller resources (as has been done for S3 Bucket recently) is that the underlying AWS API doesn't cleanly support it (unlike the S3 API which does have separate API actions for the sub-resources).
Most noticeably the attributes property is shared between the Table and any GSIs and attempting to manage this property through both an aws_dynamodb_table resource and an associated aws_dynamodb_table_gsi resource will lead to continual differences being reported by Terraform in both resources and hence to practitioner confusion.

Attempting to address this (as I see you have done) via a CustomizeDiff function on the aws_dynamodb_table is potentially brittle and a source of practitioner (and maintainer) confusion.

For example, running the GSI tests for aws_dynamodb_table now reports errors:

% make testacc TESTS=TestAccDynamoDBTable_gsi PKG=dynamodb ACCTEST_PARALLELISM=1
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/dynamodb/... -v -count 1 -parallel 1 -run='TestAccDynamoDBTable_gsi'  -timeout 180m
=== RUN   TestAccDynamoDBTable_gsiUpdateCapacity
=== PAUSE TestAccDynamoDBTable_gsiUpdateCapacity
=== RUN   TestAccDynamoDBTable_gsiUpdateOtherAttributes
=== PAUSE TestAccDynamoDBTable_gsiUpdateOtherAttributes
=== RUN   TestAccDynamoDBTable_gsiUpdateNonKeyAttributes
=== PAUSE TestAccDynamoDBTable_gsiUpdateNonKeyAttributes
=== CONT  TestAccDynamoDBTable_gsiUpdateCapacity
    table_test.go:847: Step 1/3 error: Error running apply: exit status 1
        
        Error: error creating DynamoDB Table: InvalidParameter: 1 validation error(s) found.
        - missing required field, CreateTableInput.AttributeDefinitions.
        
        
          with aws_dynamodb_table.test,
          on terraform_plugin_test.tf line 6, in resource "aws_dynamodb_table" "test":
           6: resource "aws_dynamodb_table" "test" {
        
--- FAIL: TestAccDynamoDBTable_gsiUpdateCapacity (6.15s)
=== CONT  TestAccDynamoDBTable_gsiUpdateNonKeyAttributes
    table_test.go:1044: Step 1/3 error: Error running apply: exit status 1
        
        Error: error creating DynamoDB Table: InvalidParameter: 1 validation error(s) found.
        - missing required field, CreateTableInput.AttributeDefinitions.
        
        
          with aws_dynamodb_table.test,
          on terraform_plugin_test.tf line 6, in resource "aws_dynamodb_table" "test":
           6: resource "aws_dynamodb_table" "test" {
        
--- FAIL: TestAccDynamoDBTable_gsiUpdateNonKeyAttributes (5.11s)
=== CONT  TestAccDynamoDBTable_gsiUpdateOtherAttributes
    table_test.go:915: Step 1/3 error: Error running apply: exit status 1
        
        Error: error creating DynamoDB Table: InvalidParameter: 1 validation error(s) found.
        - missing required field, CreateTableInput.AttributeDefinitions.
        
        
          with aws_dynamodb_table.test,
          on terraform_plugin_test.tf line 6, in resource "aws_dynamodb_table" "test":
           6: resource "aws_dynamodb_table" "test" {
        
--- FAIL: TestAccDynamoDBTable_gsiUpdateOtherAttributes (4.90s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/dynamodb	22.120s
FAIL
make: *** [testacc] Error 1

DynamoDB tables are mission-critical resources at many practitioners’ organizations and we prefer to err on the side of caution when making significant changes to workflows built around this resource.

An AWS support ticket has been opened to start the conversation around making changes to the DynamoDB API to enable table index operations to be performed separately from the table itself, but as you can imagine this will not be a quick change.

Any limitations to the current CustomizeDiff functionality may be mitigated by eventual migration of this resource to the terraform-plugin-framework.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.