Support Version 2 of DynamoDB Global Tables

[craig-suchanec-bw]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS recently released a new version of Global Tables for DynamoDB which changes how the the global table is defined. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html

There is an updated synchronization protocol that decreases the needed write capacity and a simplified definition of how to create a cross region replication (i.e. a global table).

Where as before to set-up cross region replication the individual tables needed to be created and a seperate globabl table to keep those tables in sync (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.tutorial.html#creategt_console) the new version API (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/V2globaltables.tutorial.html#V2creategt_cli) requires an update to the original table to tell it to replicate to separate regions rather than having separately managed tables connected through a global table. This means the current method of defining a global table (aws_dynamodb_global_table) is outdated in modeling how global tables are defined with this new API.

New or Affected Resource(s)

• aws_dynamodb_global_table
• aws_dynamodb_table

Potential Terraform Configuration

Unsure

References

• https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V2.html
• https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables.V1.html

[ewbankkit]

Existing single-region DynamoDB tables can now be converted to global tables:

• https://aws.amazon.com/blogs/aws/new-convert-your-single-region-amazon-dynamodb-tables-to-global-tables/

[crawforde]

Note that the workflow for creating and managing the v2019 tables is completely different than v2017, and that the list of available features is different. Some critical differences I've noticed include the following:

• Manual provisioning of read/write capacity on v2019 tables is no longer supported when global tables are enabled.
• Previously, replicas needed to be created first, and then the global table would be created, referencing the existing replicas. Now, a single dynamodb table is created in the "master" region, and it is subsequently updated to replicate itself to other regions.
• The method for managing TTL is different. Previously, it was a manual process to set-up a master-slave TTL relationship between existing replicas. One replica needed to have TTL enabled and the rest would not, so that TTL deletions would be initiated in one region and replicated to the others (this configuration was not enforced by the AWS cli, but it was an AWS recommended best practice). Now, the concept of master-slave is a built-in part of v2019 tables, so when TTL is enabled on a table, all replicas are marked as having TTL "enabled" (although, behind the scenes the same behavior as before is still taking place).
• New regions can now be added to the table without removing all data from it.

Given the differences listed above, there may be an argument here for implementing this new feature as a completely separate resource type, aws_dynamodb_table_v2019 or something along those lines.

This tutorial is a good reference for the new workflow: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/V2globaltables.tutorial.html#V2creategt_cli

[crawforde]

Existing single-region DynamoDB tables can now be converted to global tables:

• https://aws.amazon.com/blogs/aws/new-convert-your-single-region-amazon-dynamodb-tables-to-global-tables/

NOTE: this describes the behavior of the new v2019 tables, but I'm fairly certain it does not imply that existing v2017 single-region tables can be converted (please correct me if you find documentation to the contrary). One thing that I am sure about is that existing v2017 multi-region tables cannot be converted to v2019.

[cmonty]

@crawforde I took a stab at implementing the new version based on your suggestions (we have a need for this functionality). I'm not sure it should be a separate resource, though, as it's my understanding if you have a DynamoDB table that is not currently being replicated as a global table (V2017) you can start replicating it using V2019 (AFAIK, non-global tables don't have a V2017/V2019 version).

Let me know what you think.

[stolfi]

I'm doing some testing on your PR now. The only issue I've seen is that it's wanting to recreate all of my GSIs on two of my tables.... This appears separate from the other issue of recreating GSIs, as some of them don't have any non_key_attributes.

[cmonty]

@stolfi hmm...is it related to billing_mode?

[stolfi]

@stolfi hmm...is it related to billing_mode?

I'm currently PPR, let me move that table to provisioned to see if it makes a difference. I have 17 tables, so it's odd that only two have this issue.

[stolfi]

Sorry for the delay... It does appear to be billing mode related. I moved to PROVISIONED on one of two tables and am able to get the plan to come back with no changes.

Let me see if I can hunt that down and put in a fix.

[cmonty]

@stolfi awesome. Thanks for testing!

[bflad]

Initial support for this functionality (a new replica configuration block in the aws_dynamodb_table resource) has been merged and will release with version 2.58.0 of the Terraform AWS Provider, later this week. Thanks to @cmonty for the implementation. 👍

[ghost]

This has been released in version 2.58.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!