Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/aws_rds_cluster: Prevent restored cluster recreation with kms_key_id and lack of storage_encrypted, add testing and documentation #15915

Merged
merged 1 commit into from
Nov 5, 2020

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Oct 29, 2020

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #3503

Release note for CHANGELOG:

* resource/aws_rds_cluster: Prevent recreation when using `snapshot_identifier` and `kms_key_id` without `storage_encrypted = true`

Please note that previously restoring an unencrypted snapshot_identifier did create an encrypted cluster with kms_key_id argument, it just also required storage_encryption = true to prevent resource recreation immediately afterwards. We intend to apply similar fixes to other RDS resources, such as aws_rds_global_cluster for restore situations like these.

Output from acceptance testing:

--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_KmsKeyId (395.41s)

…s_key_id and lack of storage_encrypted, add testing and documentation

Reference: #3503

Output from acceptance testing:

```
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_KmsKeyId (395.41s)
```
@bflad bflad added the bug Addresses a defect in current functionality. label Oct 29, 2020
@bflad bflad requested a review from a team October 29, 2020 18:07
@ghost ghost added size/M Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/rds Issues and PRs that pertain to the rds service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 29, 2020
@bflad
Copy link
Contributor Author

bflad commented Oct 30, 2020

I'm working on separately fixing these problematic tests:

--- FAIL: TestAccAWSRDSCluster_EngineMode_Global (131.39s)
--- FAIL: TestAccAWSRDSCluster_EngineVersionWithPrimaryInstance (1058.15s)
--- FAIL: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global (133.74s)
--- FAIL: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Add (130.51s)
--- FAIL: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Remove (123.68s)
--- FAIL: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Update (123.65s)

@bflad
Copy link
Contributor Author

bflad commented Oct 30, 2020

See #15938 for those unrelated testing issues and fixes.

@anGie44 anGie44 self-assigned this Nov 5, 2020
Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change here LGTM 👍

Output of tests (ignoring global cluster failures):

-- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql1 (627.58s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1 (628.78s)
--- PASS: TestAccAWSRDSClusterInstance_generatedName (690.08s)
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (690.29s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraPostgresql (693.87s)
--- PASS: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (714.93s)
--- PASS: TestAccAWSRDSClusterInstance_az (737.04s)
--- PASS: TestAccAWSRDSClusterInstance_isAlreadyBeingDeleted (745.06s)
--- PASS: TestAccAWSRDSClusterInstance_disappears (755.53s)
--- PASS: TestAccAWSRDSClusterInstance_PubliclyAccessible (757.60s)
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (757.81s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsEnabled_AuroraMysql2 (784.10s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1_DefaultKeyToCustomKey (784.37s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_RemovedToEnabled (803.63s)
--- PASS: TestAccAWSRDSCluster_basic (125.89s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToDisabled (843.39s)
--- PASS: TestAccAWSRDSCluster_missingUserNameCausesError (5.18s)
--- PASS: TestAccAWSRDSCluster_AvailabilityZones (116.76s)
--- PASS: TestAccAWSRDSCluster_ClusterIdentifierPrefix (126.89s)
--- PASS: TestAccAWSRDSCluster_DbSubnetGroupName (130.68s)
--- PASS: TestAccAWSRDSCluster_generatedName (126.12s)
--- PASS: TestAccAWSRDSCluster_BacktrackWindow (168.55s)
--- PASS: TestAccAWSRDSCluster_takeFinalSnapshot (166.09s)
--- PASS: TestAccAWSRDSCluster_EnabledCloudwatchLogsExports_Postgresql (125.93s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToRemoved (991.15s)
--- PASS: TestAccAWSRDSCluster_Tags (147.62s)
--- PASS: TestAccAWSRDSCluster_kmsKey (126.16s)
--- PASS: TestAccAWSRDSCluster_encrypted (117.36s)
--- PASS: TestAccAWSRDSCluster_updateIamRoles (144.78s)
--- PASS: TestAccAWSRDSClusterEndpoint_basic (1036.83s)
--- PASS: TestAccAWSRDSCluster_EnabledCloudwatchLogsExports_MySQL (208.97s)
--- PASS: TestAccAWSRDSCluster_iamAuth (127.66s)
--- PASS: TestAccAWSRDSCluster_EngineMode_Multimaster (130.78s)
--- PASS: TestAccAWSRDSCluster_copyTagsToSnapshot (240.73s)
--- PASS: TestAccAWSRDSCluster_backupsUpdate (188.38s)
--- PASS: TestAccAWSRDSClusterEndpoint_tags (1177.50s)
--- PASS: TestAccAWSRDSCluster_DeletionProtection (188.44s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql2 (572.87s)
--- PASS: TestAccAWSRDSCluster_EngineMode_ParallelQuery (166.50s)
--- PASS: TestAccAWSRDSClusterInstance_CACertificateIdentifier (513.05s)
--- PASS: TestAccAWSRDSClusterInstance_MonitoringInterval (1293.05s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Provisioned (149.16s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraPostgresql (657.29s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql2_DefaultKeyToCustomKey (741.32s)
--- PASS: TestAccAWSRDSCluster_Port (248.25s)
--- PASS: TestAccAWSRDSCluster_EngineMode (441.91s)
--- PASS: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraPostgresql_DefaultKeyToCustomKey (773.99s)
--- PASS: TestAccAWSRDSClusterInstance_basic (1480.61s)
--- PASS: TestAccAWSRDSCluster_EngineVersion (443.13s)
--- PASS: TestAccAWSRDSCluster_ScalingConfiguration (368.54s)
--- PASS: TestAccAWSRDSCluster_ScalingConfiguration_DefaultMinCapacity (307.09s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier (342.72s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineMode_Provisioned (363.55s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_DeletionProtection (394.98s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineVersion_Different (404.23s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineVersion_Equal (384.59s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineMode_ParallelQuery (423.97s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_KmsKeyId (364.32s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_MasterUsername (355.28s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_PreferredBackupWindow (354.46s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_MasterPassword (384.89s)
--- PASS: TestAccAWSRDSCluster_AllowMajorVersionUpgrade (1139.91s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_PreferredMaintenanceWindow (413.84s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_Tags (423.63s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_VpcSecurityGroupIds (384.93s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_VpcSecurityGroupIds_Tags (374.78s)
--- PASS: TestAccAWSRDSCluster_EnableHttpEndpoint (358.44s)
--- PASS: TestAccAWSRDSCluster_EngineVersionWithPrimaryInstance (1157.33s)
--- PASS: TestAccAWSRDSCluster_ReplicationSourceIdentifier_KmsKeyId (1486.16s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_PrimarySecondaryClusters (1691.40s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier (1755.41s)

@bflad bflad added this to the v3.14.0 milestone Nov 5, 2020
@bflad bflad merged commit e0f12df into master Nov 5, 2020
@bflad bflad deleted the b-aws_rds_cluster-storage_encrypted branch November 5, 2020 23:43
bflad added a commit that referenced this pull request Nov 5, 2020
@ghost
Copy link

ghost commented Nov 6, 2020

This has been released in version 3.14.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@callaingit
Copy link

Used it and now the delete with a secondary cluster went fine! Nice.

@ghost
Copy link

ghost commented Dec 6, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Dec 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. documentation Introduces or discusses updates to documentation. service/rds Issues and PRs that pertain to the rds service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to create an encrypted Aurora cluster from a non-encrypted snapshot
3 participants