Allow adding member Replication Group to ElastiCache Global Replication Group #17725

gdavison · 2021-02-19T22:53:44Z

PR #17615 added the aws_elasticache_global_replication_group resource, but only the primary Replication Group could be added. This PR extends aws_elasticache_replication_group to allow adding secondary Replication Groups to the Global Replication Group.

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #12708
Closes #17812

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSElasticacheGlobalReplicationGroup_\|TestAccAWSElasticacheReplicationGroup_'

--- PASS: TestAccAWSElasticacheReplicationGroup_Validation_NoNodeType (84.85s)
--- PASS: TestAccAWSElasticacheReplicationGroup_vpc (814.98s)
--- PASS: TestAccAWSElasticacheReplicationGroup_tags (1029.32s)
--- PASS: TestAccAWSElasticacheReplicationGroup_Uppercase (1096.03s)
--- PASS: TestAccAWSElasticacheReplicationGroup_FinalSnapshot (1480.67s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_basic (1542.92s)
--- PASS: TestAccAWSElasticacheReplicationGroup_updateDescription (1847.23s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_AddMemberCluster (1893.50s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_RemoveMemberCluster_AtTargetSize (1981.46s)
--- PASS: TestAccAWSElasticacheReplicationGroup_multiAz_NoAutomaticFailover (8.26s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_RemoveMemberCluster_ScaleDown (2013.37s)
--- PASS: TestAccAWSElasticacheReplicationGroup_updateNodeSize (2013.81s)
--- PASS: TestAccAWSElasticacheReplicationGroup_AutomaticFailover_OneCacheCluster (13.35s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_NoChange (2038.90s)
--- PASS: TestAccAWSElasticacheReplicationGroup_basic (2160.00s)
--- PASS: TestAccAWSElasticacheReplicationGroup_disappears (2239.66s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_Basic (1537.42s)
--- PASS: TestAccAWSElasticacheReplicationGroup_updateParameterGroup (2617.01s)
--- PASS: TestAccAWSElasticacheReplicationGroup_updateMaintenanceWindow (2682.82s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_Failover_AutoFailoverEnabled (1670.43s)
--- PASS: TestAccAWSElasticacheReplicationGroup_useCmkKmsKeyId (714.18s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_NonClusteredParameterGroup (942.23s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MultiAZEnabled (2154.49s)
--- PASS: TestAccAWSElasticacheReplicationGroup_Validation_GlobalReplicationGroupIdAndNodeType (1549.39s)
--- PASS: TestAccAWSElasticacheReplicationGroup_clusteringAndCacheNodesCausesError (10.23s)
--- PASS: TestAccAWSElasticacheReplicationGroup_enableAuthTokenTransitEncryption (946.09s)
--- PASS: TestAccAWSElasticacheReplicationGroup_redisClusterInVpc2 (1300.66s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroupsAndReplicasPerNodeGroup_ScaleUp (3400.13s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateReplicasPerNodeGroup (2008.40s)
--- PASS: TestAccAWSElasticacheReplicationGroup_enableAtRestEncryption (812.97s)
--- PASS: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_disappears (3686.90s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_Description (1825.51s)
--- PASS: TestAccAWSElasticacheReplicationGroup_multiAzInVpc (1056.03s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_Failover_AutoFailoverDisabled (1369.65s)
--- PASS: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_Basic (3859.05s)
--- PASS: TestAccAWSElasticacheReplicationGroup_enableSnapshotting (1023.18s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroups_ScaleDown (2142.82s)
--- PASS: TestAccAWSElasticacheReplicationGroup_multiAzNotInVpc (2182.78s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_disappears (1690.90s)
--- PASS: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_Basic (1548.16s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroups_ScaleUp (2686.88s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_ReplaceSecondary_DifferentRegion (4857.02s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_MultipleSecondaries (5230.23s)
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroupsAndReplicasPerNodeGroup_ScaleDown (2868.77s)

YakDriver · 2021-03-10T17:25:18Z

@gdavison I'm hitting failures:

=== CONT  TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_disappears
    resource_aws_elasticache_replication_group_test.go:1447: Step 1/1 error: Error running apply: exit status 1
        
        Error: Error creating ElastiCache Replication Group (tf-acc-test-4593258471161770688-p): InvalidParameterCombination: Instance type cache.m5.large can only be created in a VPC.
        	status code: 400, request id: 7c51687f-8ed7-44ee-9d09-d061d2f3e2bb
        
          on terraform_plugin_test.tf line 21, in resource "aws_elasticache_replication_group" "primary":
          21: resource "aws_elasticache_replication_group" "primary" {
        
        
--- FAIL: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_disappears (11.57s)
=== CONT  TestAccAWSElasticacheReplicationGroup_enableAtRestEncryption
=== CONT  TestAccAWSElasticacheReplicationGroup_Validation_GlobalReplicationGroupIdAndNodeType
    resource_aws_elasticache_replication_group_test.go:1388: Step 1/1, expected an error with pattern, no match on: Error running apply: exit status 1
        
        Error: Error creating ElastiCache Replication Group (tf-acc-test-6342892834211265252-p): InvalidParameterCombination: Instance type cache.m5.large can only be created in a VPC.
        	status code: 400, request id: 19ba5c09-15c0-4371-844e-09b6cfc79141
        
          on terraform_plugin_test.tf line 23, in resource "aws_elasticache_replication_group" "primary":
          23: resource "aws_elasticache_replication_group" "primary" {
        
        
=== CONT  TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_Basic
    resource_aws_elasticache_replication_group_test.go:1410: Step 1/2 error: Error running apply: exit status 1
        
        Error: Error creating ElastiCache Replication Group (tf-acc-test-5493595774239971302-p): InvalidParameterCombination: Instance type cache.m5.large can only be created in a VPC.
        	status code: 400, request id: eb38bc58-8edb-4350-8adc-b53d6dee8750
        
          on terraform_plugin_test.tf line 21, in resource "aws_elasticache_replication_group" "primary":
          21: resource "aws_elasticache_replication_group" "primary" {
        
        
--- FAIL: TestAccAWSElasticacheReplicationGroup_Validation_GlobalReplicationGroupIdAndNodeType (12.06s)
=== CONT  TestAccAWSElasticacheReplicationGroup_enableAuthTokenTransitEncryption
--- FAIL: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_Basic (12.29s)
=== CONT  TestAccAWSElasticacheReplicationGroup_enableSnapshotting
=== CONT  TestAccAWSElasticacheReplicationGroup_clusteringAndCacheNodesCausesError
=== CONT  TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroupsAndReplicasPerNodeGroup_ScaleDown
=== CONT  TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_NoChange
    resource_aws_elasticache_replication_group_test.go:1123: Step 1/2 error: Error running post-apply refresh: exit status 1
        
        Error: error reading EC2 VPC (vpc-0b761d64697108d32) Attribute (enableDnsSupport): RequestError: send request failed
        caused by: Post "https://ec2.us-west-2.amazonaws.com/": read tcp 192.168.0.190:60162->54.240.253.45:443: read: connection reset by peer
        
        
=== CONT  TestAccAWSElasticacheReplicationGroup_enableAtRestEncryption
    resource_aws_elasticache_replication_group_test.go:862: Step 1/2 error: Error running apply: exit status 1
        
        Error: error creating ElastiCache Replication Group (tf-hnkzi023ze): waiting for completion: RequestError: send request failed
        caused by: Post "https://elasticache.us-west-2.amazonaws.com/": read tcp 192.168.0.190:60166->52.94.208.134:443: read: connection reset by peer
        
          on terraform_plugin_test.tf line 51, in resource "aws_elasticache_replication_group" "test":
          51: resource "aws_elasticache_replication_group" "test" {
        
        
=== CONT  TestAccAWSElasticacheReplicationGroup_updateNodeSize
    resource_aws_elasticache_replication_group_test.go:240: Step 3/3 error: Error running apply: exit status 1
        
        Error: error updating ElastiCache Replication Group (tf-acc-test-3970208893006979190): error waiting for modification: RequestError: send request failed
        caused by: Post "https://elasticache.us-west-2.amazonaws.com/": read tcp 192.168.0.190:60168->52.94.208.134:443: read: connection reset by peer
        
          on terraform_plugin_test.tf line 2, in resource "aws_elasticache_replication_group" "test":
           2: resource "aws_elasticache_replication_group" "test" {
        
        
=== CONT  TestAccAWSElasticacheReplicationGroup_enableAuthTokenTransitEncryption
    resource_aws_elasticache_replication_group_test.go:835: Step 1/2 error: Error running apply: exit status 1
        
        Error: error creating ElastiCache Replication Group (tf-i1c2l6v4dk): waiting for completion: SignatureDoesNotMatch: Signature expired: 20210310T155102Z is now earlier than 20210310T155303Z (20210310T160803Z - 15 min.)
        	status code: 403, request id: 6d56b9b9-15d7-49f7-9209-b7b30bad2c27
        
          on terraform_plugin_test.tf line 51, in resource "aws_elasticache_replication_group" "test":
          51: resource "aws_elasticache_replication_group" "test" {
        
        
=== CONT  TestAccAWSElasticacheReplicationGroup_tags
    testing_new.go:63: Error running post-test destroy, there may be dangling resources: exit status 1
        
        Error: error deleting ElastiCache Replication Group (tf-acc-test-1148291201744263816): SignatureDoesNotMatch: Signature expired: 20210310T155102Z is now earlier than 20210310T155303Z (20210310T160803Z - 15 min.)
        	status code: 403, request id: 13db9f45-9ce8-48e1-a690-ca53ac45af9c
        
        
--- FAIL: TestAccAWSElasticacheReplicationGroup_tags (766.96s)
=== CONT  TestAccAWSElasticacheReplicationGroup_vpc
=== CONT  TestAccAWSElasticacheReplicationGroup_multiAzNotInVpc
=== CONT  TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroupsAndReplicasPerNodeGroup_ScaleUp
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_basic
    resource_aws_elasticache_global_replication_group_test.go:82: Step 1/2 error: Error running apply: exit status 1
        
        Error: error waiting for ElastiCache Global Replication Group (sgaui-tf-acc-test-7153180625914869287) availability: RequestError: send request failed
        caused by: Post "https://elasticache.us-west-2.amazonaws.com/": read tcp 192.168.0.190:60161->52.94.208.134:443: read: connection reset by peer
        
          on terraform_plugin_test.tf line 2, in resource "aws_elasticache_global_replication_group" "test":
           2: resource "aws_elasticache_global_replication_group" "test" {
        
        
--- FAIL: TestAccAWSElasticacheReplicationGroup_enableAtRestEncryption (1092.31s)
=== CONT  TestAccAWSElasticacheReplicationGroup_basic
--- PASS: TestAccAWSElasticacheReplicationGroup_redisClusterInVpc2 (1115.59s)
=== CONT  TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateReplicasPerNodeGroup
--- FAIL: TestAccAWSElasticacheReplicationGroup_updateNodeSize (1153.85s)
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_MultipleSecondaries
--- PASS: TestAccAWSElasticacheReplicationGroup_enableSnapshotting (1146.88s)
=== CONT  TestAccAWSElasticacheReplicationGroup_updateMaintenanceWindow
--- PASS: TestAccAWSElasticacheReplicationGroup_ClusterMode_Basic (1177.92s)
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_ReplaceSecondary_DifferentRegion
--- FAIL: TestAccAWSElasticacheReplicationGroup_enableAuthTokenTransitEncryption (1206.97s)
=== CONT  TestAccAWSElasticacheReplicationGroup_updateDescription
=== CONT  TestAccAWSElasticacheReplicationGroup_Uppercase
--- FAIL: TestAccAWSElasticacheReplicationGroup_NumberCacheClusters_MemberClusterDisappears_NoChange (1279.66s)
=== CONT  TestAccAWSElasticacheReplicationGroup_ClusterMode_UpdateNumNodeGroups_ScaleDown
=== CONT  TestAccAWSElasticacheReplicationGroup_updateParameterGroup
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_disappears
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_Description
=== CONT  TestAccAWSElasticacheReplicationGroup_disappears
--- FAIL: TestAccAWSElasticacheGlobalReplicationGroup_basic (1713.36s)
=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_ReplaceSecondary_DifferentRegion
    resource_aws_elasticache_global_replication_group_test.go:204: Step 1/2 error: Error running apply: exit status 1
        
        Error: Error creating ElastiCache Replication Group (tf-acc-test-373891496539671179-a): InvalidParameterValue: This operation is only supported for VPC replication groups.
        	status code: 400, request id: 35ebcbc7-1a5d-4422-adb0-4ca643389a86
        
          on terraform_plugin_test.tf line 30, in resource "aws_elasticache_replication_group" "secondary":
          30: resource "aws_elasticache_replication_group" "secondary" {

=== CONT  TestAccAWSElasticacheGlobalReplicationGroup_MultipleSecondaries
    resource_aws_elasticache_global_replication_group_test.go:180: Step 1/1 error: Error running apply: exit status 1
        
        Error: Error creating ElastiCache Replication Group (tf-acc-test-9018238266104004182-a): InvalidParameterValue: This operation is only supported for VPC replication groups.
        	status code: 400, request id: 829c6778-8f7b-4940-994b-31aa8bdb132c
        
          on terraform_plugin_test.tf line 30, in resource "aws_elasticache_replication_group" "alternate":
          30: resource "aws_elasticache_replication_group" "alternate" {
 
--- FAIL: TestAccAWSElasticacheGlobalReplicationGroup_ReplaceSecondary_DifferentRegion (2003.29s)
--- FAIL: TestAccAWSElasticacheGlobalReplicationGroup_MultipleSecondaries (3865.51s)

YakDriver

Looks good! 🎉

I ran all tests and they passed (closed the window with most of the results and won't run them again just for that text).

Problematic acceptance tests in commercial (us-west-2):

--- PASS: TestAccAWSElasticacheReplicationGroup_Validation_GlobalReplicationGroupIdAndNodeType (1702.14s)
--- PASS: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_Basic (3475.22s)
--- PASS: TestAccAWSElasticacheReplicationGroup_GlobalReplicationGroupId_disappears (3475.41s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_MultipleSecondaries (4702.57s)
--- PASS: TestAccAWSElasticacheGlobalReplicationGroup_ReplaceSecondary_DifferentRegion (5296.17s)

YakDriver · 2021-03-11T15:46:52Z

aws/internal/service/elasticache/waiter/waiter.go

+	GlobalReplicationGroupDisassociationRetryTimeout = 45 * time.Minute
+
+	globalReplicationGroupDisassociationTimeout = 20 * time.Minute
+
+	globalReplicationGroupDisassociationMinTimeout = 10 * time.Second
+	globalReplicationGroupDisassociationDelay      = 30 * time.Second


Suggested change

GlobalReplicationGroupDisassociationRetryTimeout = 45 * time.Minute

globalReplicationGroupDisassociationTimeout = 20 * time.Minute

globalReplicationGroupDisassociationMinTimeout = 10 * time.Second

globalReplicationGroupDisassociationDelay = 30 * time.Second

GlobalReplicationGroupDisassociationRetryTimeout = 45 * time.Minute

GlobalReplicationGroupDisassociationTimeout = 20 * time.Minute

GlobalReplicationGroupDisassociationMinTimeout = 10 * time.Second

GlobalReplicationGroupDisassociationDelay = 30 * time.Second

Small nit: I see no downside in exporting all the constants. With this naming scheme, collisions are unlikely.

YakDriver · 2021-03-11T15:47:48Z

aws/internal/service/elasticache/waiter/waiter.go

+		Timeout:    globalReplicationGroupDisassociationTimeout,
+		MinTimeout: globalReplicationGroupDisassociationMinTimeout,
+		Delay:      globalReplicationGroupDisassociationDelay,


Suggested change

Timeout: globalReplicationGroupDisassociationTimeout,

MinTimeout: globalReplicationGroupDisassociationMinTimeout,

Delay: globalReplicationGroupDisassociationDelay,

Timeout: GlobalReplicationGroupDisassociationTimeout,

MinTimeout: GlobalReplicationGroupDisassociationMinTimeout,

Delay: GlobalReplicationGroupDisassociationDelay,

ghost · 2021-03-12T01:00:23Z

This has been released in version 3.32.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2021-04-11T17:10:59Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

gdavison added 5 commits February 19, 2021 09:48

Adds ElastiCache Replication Group ID validator

54338ca

Allows creating replication group as member of global replication group

f8ce42d

Updates documentation

ba0eda6

Adds tests for adding members to global replication groups

b9fa4c4

Removes unused code

9234996

gdavison requested a review from a team as a code owner February 19, 2021 22:53

bflad mentioned this pull request Mar 1, 2021

Can’t assign a secondary elasticache replication group to a global elasticache replication group #17812

Closed

gdavison added 2 commits March 9, 2021 11:51

Merge branch 'main' into elasticache_global_replication_group_member

21eafb1

Uses nosemgrep to exclude matches

6ff6cd0

YakDriver self-assigned this Mar 10, 2021

Merge branch 'main' into elasticache_global_replication_group_member

ee26ba8

ghost added size/XXL Managed by automation to categorize the size of a PR. and removed size/XL Managed by automation to categorize the size of a PR. labels Mar 11, 2021

Adds VPC to Global Replication Group tests

f5dc2d0

gdavison force-pushed the elasticache_global_replication_group_member branch from fd339bc to f5dc2d0 Compare March 11, 2021 20:31

ghost added size/XL Managed by automation to categorize the size of a PR. provider Pertains to the provider itself, rather than any interaction with AWS. and removed size/XXL Managed by automation to categorize the size of a PR. labels Mar 11, 2021

YakDriver approved these changes Mar 11, 2021

View reviewed changes

Adds CHANGELOG

6119268

gdavison merged commit b76e998 into main Mar 11, 2021

gdavison deleted the elasticache_global_replication_group_member branch March 11, 2021 23:56

github-actions bot added this to the v3.32.0 milestone Mar 11, 2021

github-actions bot pushed a commit that referenced this pull request Mar 11, 2021

Update CHANGELOG.md for #17725

1323135

anGie44 mentioned this pull request Mar 12, 2021

Secondary Elasticache Cluster will get recreated after each apply #18075

Closed

ghost locked as resolved and limited conversation to collaborators Apr 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow adding member Replication Group to ElastiCache Global Replication Group #17725

Allow adding member Replication Group to ElastiCache Global Replication Group #17725

gdavison commented Feb 19, 2021 •

edited

Loading

YakDriver commented Mar 10, 2021

YakDriver left a comment

YakDriver Mar 11, 2021

YakDriver Mar 11, 2021

ghost commented Mar 12, 2021

ghost commented Apr 11, 2021

Allow adding member Replication Group to ElastiCache Global Replication Group #17725

Allow adding member Replication Group to ElastiCache Global Replication Group #17725

Conversation

gdavison commented Feb 19, 2021 • edited Loading

Community Note

YakDriver commented Mar 10, 2021

YakDriver left a comment

Choose a reason for hiding this comment

YakDriver Mar 11, 2021

Choose a reason for hiding this comment

YakDriver Mar 11, 2021

Choose a reason for hiding this comment

ghost commented Mar 12, 2021

ghost commented Apr 11, 2021

gdavison commented Feb 19, 2021 •

edited

Loading