Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynamoDB Global table V2 - Creation fails with multiple replicas #13132

Closed
giacomomagini opened this issue May 2, 2020 · 9 comments · Fixed by #13523
Closed

DynamoDB Global table V2 - Creation fails with multiple replicas #13132

giacomomagini opened this issue May 2, 2020 · 9 comments · Fixed by #13523
Assignees
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Milestone

Comments

@giacomomagini
Copy link

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.12.24

Affected Resource(s)

  • aws_dynamodb_table

Terraform Configuration Files

provider "aws" {
  region = "eu-west-1"
}

resource "aws_dynamodb_table" "t" {

  name     = "test-table"
  hash_key = "key"

  stream_enabled   = true
  stream_view_type = "NEW_AND_OLD_IMAGES"

  billing_mode = "PAY_PER_REQUEST"

  attribute {
    name = "key"
    type = "S"
  }

  replica {
    region_name = "us-east-1"
  }

  replica {
    region_name = "us-east-2"
  }

}

Debug Output

https://gist.github.com/giacomomagini/4e308947a2ad7c857ecb2edc3961dbaa

Panic Output

Expected Behavior

Create a global table with the two replicas us-east-1 and us-east-2.

Actual Behavior

Error: error creating DynamoDB Table (test-table) replicas: error creating DynamoDB Table (test-table) replicas: ValidationException: Update table operation with more than one create or delete replica actions not allowed

Steps to Reproduce

  1. terraform init
  2. terraform plan -out=plan.out <module_path>
  3. terraform apply plan.out

Important Factoids

None.

References

The first implementation of the code is in this PR (#12342) but a refactor has been done afterwards in this commit (715b453)

The issue is that tf sends a single update with all the replica create commands while it should send these every create command in a different call.

  • #0000
@ghost ghost added the service/dynamodb Issues and PRs that pertain to the dynamodb service. label May 2, 2020
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label May 2, 2020
@joshmyers
Copy link
Contributor

I wonder if this worked in the original implementation but broke in 715b453#diff-c8928c2dd5459e42d28428384a191ba8L633 @cmonty / @bflad

There aren't currently any tests to cover multiple replicas.

@cmonty
Copy link
Contributor

cmonty commented May 16, 2020

We did test creating replicas with the original implementation and that worked. I don't see any specific documentation from AWS on this topic.

@jsalcedo1987
Copy link

I'm facing the same issue with Terraform and Dynamo Global Tables.
Is there any workaround available?

@giacomomagini
Copy link
Author

@jsalcedo1987 the workaround is adding the replicas one by one. Add, plan, apply for every single replica.

@giacomomagini
Copy link
Author

@cmonty the documentation is not clear at all but if you look at the tutorial they tell you to run a command for every replica you want to create.
image
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/V2globaltables.tutorial.html

@bflad bflad added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels May 27, 2020
@bflad bflad self-assigned this May 27, 2020
bflad added a commit that referenced this issue May 27, 2020
…etions

Reference: #13132

Also includes slight changes to the acceptance testing recommendations for cross-region testing since we now are introducing the framework to test a third region. If approved, will followup this change with refactoring of the existing testing to use the newer functions.

Previously:

```
--- FAIL: TestAccAWSDynamoDbTable_Replica_Multiple (41.02s)
    TestAccAWSDynamoDbTable_Replica_Multiple: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DynamoDB Table (TerraformTestTable--8440396559818741779) replicas: error creating DynamoDB Table (TerraformTestTable--8440396559818741779) replicas: ValidationException: Update table operation with more than one create or delete replica actions not allowed
          status code: 400, request id: 5MA0CQ5LJAVOKM671RT7QTKRTJVV4KQNSO5AEMVJF66Q9ASUAAJG

          on /var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/tf-test945359260/main.tf line 20:
          (source code not available)
```

Output from acceptance testing:

```
--- PASS: TestAccAWSDynamoDbTable_attributeUpdate (756.09s)
--- PASS: TestAccAWSDynamoDbTable_attributeUpdateValidation (17.31s)
--- PASS: TestAccAWSDynamoDbTable_basic (82.00s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_PayPerRequestToProvisioned (135.99s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_ProvisionedToPayPerRequest (1296.50s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_PayPerRequestToProvisioned (134.93s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_ProvisionedToPayPerRequest (934.09s)
--- PASS: TestAccAWSDynamoDbTable_disappears (59.40s)
--- PASS: TestAccAWSDynamoDbTable_disappears_PayPerRequestWithGSI (140.30s)
--- PASS: TestAccAWSDynamoDbTable_enablePitr (152.34s)
--- PASS: TestAccAWSDynamoDbTable_encryption (249.98s)
--- PASS: TestAccAWSDynamoDbTable_extended (443.38s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateCapacity (144.47s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateNonKeyAttributes (443.69s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateOtherAttributes (757.61s)
--- PASS: TestAccAWSDynamoDbTable_Replica_Multiple (1193.16s)
--- PASS: TestAccAWSDynamoDbTable_Replica_Single (1077.60s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecification (148.25s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecificationValidation (12.21s)
--- PASS: TestAccAWSDynamoDbTable_tags (88.24s)
--- PASS: TestAccAWSDynamoDbTable_Ttl_Disabled (116.66s)
--- PASS: TestAccAWSDynamoDbTable_Ttl_Enabled (81.82s)
```
@bflad
Copy link
Contributor

bflad commented May 27, 2020

Sorry for the hassle here folks 😓 that followup optimization attempt should have been proven via additional acceptance testing. To that end, have submitted the following to add the testing and fix this issue: #13523

bflad added a commit that referenced this issue Jun 3, 2020
…etions (#13523)

* resource/aws_dynamodb_table: Perform individual replica creations/deletions

Reference: #13132

Also includes slight changes to the acceptance testing recommendations for cross-region testing since we now are introducing the framework to test a third region. If approved, will followup this change with refactoring of the existing testing to use the newer functions.

Previously:

```
--- FAIL: TestAccAWSDynamoDbTable_Replica_Multiple (41.02s)
    TestAccAWSDynamoDbTable_Replica_Multiple: testing.go:684: Step 0 error: errors during apply:

        Error: error creating DynamoDB Table (TerraformTestTable--8440396559818741779) replicas: error creating DynamoDB Table (TerraformTestTable--8440396559818741779) replicas: ValidationException: Update table operation with more than one create or delete replica actions not allowed
          status code: 400, request id: 5MA0CQ5LJAVOKM671RT7QTKRTJVV4KQNSO5AEMVJF66Q9ASUAAJG

          on /var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/tf-test945359260/main.tf line 20:
          (source code not available)
```

Output from acceptance testing:

```
--- PASS: TestAccAWSDynamoDbTable_attributeUpdate (756.09s)
--- PASS: TestAccAWSDynamoDbTable_attributeUpdateValidation (17.31s)
--- PASS: TestAccAWSDynamoDbTable_basic (82.00s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_PayPerRequestToProvisioned (135.99s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_GSI_ProvisionedToPayPerRequest (1296.50s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_PayPerRequestToProvisioned (134.93s)
--- PASS: TestAccAWSDynamoDbTable_BillingMode_ProvisionedToPayPerRequest (934.09s)
--- PASS: TestAccAWSDynamoDbTable_disappears (59.40s)
--- PASS: TestAccAWSDynamoDbTable_disappears_PayPerRequestWithGSI (140.30s)
--- PASS: TestAccAWSDynamoDbTable_enablePitr (152.34s)
--- PASS: TestAccAWSDynamoDbTable_encryption (249.98s)
--- PASS: TestAccAWSDynamoDbTable_extended (443.38s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateCapacity (144.47s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateNonKeyAttributes (443.69s)
--- PASS: TestAccAWSDynamoDbTable_gsiUpdateOtherAttributes (757.61s)
--- PASS: TestAccAWSDynamoDbTable_Replica_Multiple (1193.16s)
--- PASS: TestAccAWSDynamoDbTable_Replica_Single (1077.60s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecification (148.25s)
--- PASS: TestAccAWSDynamoDbTable_streamSpecificationValidation (12.21s)
--- PASS: TestAccAWSDynamoDbTable_tags (88.24s)
--- PASS: TestAccAWSDynamoDbTable_Ttl_Disabled (116.66s)
--- PASS: TestAccAWSDynamoDbTable_Ttl_Enabled (81.82s)
```

* tests/provider: Consolidate logic into testAccMultipleRegionPreCheck
@bflad bflad added this to the v2.65.0 milestone Jun 3, 2020
@bflad
Copy link
Contributor

bflad commented Jun 3, 2020

The fix for this has been merged, verified, and will release with version 2.65.0 of the Terraform AWS Provider, likely tomorrow.

@ghost
Copy link

ghost commented Jun 5, 2020

This has been released in version 2.65.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Jul 4, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jul 4, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Projects
None yet
5 participants