r/aws_db_instance: Fix error modifying `allocated_storage` when `storage_type` is `"gp3"` #28243

ewbankkit · 2022-12-07T22:15:00Z

Description

Prevents errors like

 Error: updating RDS DB Instance (tf-acc-test-5977403606277891875): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 6086d274-e75b-445c-afc5-8f6b14bbba02, api error InvalidParameterCombination: You can't specify IOPS or storage throughput for engine mysql and a storage size less than 400.

modifying allocated_storage when storage_type is "gp3" and allocated_storage is below a per-engine threshold.

Relations

Closes #28160.

Output from Acceptance Testing

% make testacc TESTARGS='-run=TestAccRDSInstance_storageTypePostgres' PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_storageTypePostgres -timeout 180m
=== RUN   TestAccRDSInstance_storageTypePostgres
=== PAUSE TestAccRDSInstance_storageTypePostgres
=== CONT  TestAccRDSInstance_storageTypePostgres
--- PASS: TestAccRDSInstance_storageTypePostgres (796.18s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/rds	801.356s
% make testacc TESTARGS='-run=TestAccRDSInstance_gp3SQLServer' PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_gp3SQLServer -timeout 180m
=== RUN   TestAccRDSInstance_gp3SQLServer
=== PAUSE TestAccRDSInstance_gp3SQLServer
=== CONT  TestAccRDSInstance_gp3SQLServer
--- PASS: TestAccRDSInstance_gp3SQLServer (1061.41s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/rds	1066.408s

% make testacc TESTARGS='-run=TestAccRDSInstance_storageThroughput' PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_storageThroughput -timeout 180m
=== RUN   TestAccRDSInstance_storageThroughput
=== PAUSE TestAccRDSInstance_storageThroughput
=== CONT  TestAccRDSInstance_storageThroughput
--- PASS: TestAccRDSInstance_storageThroughput (783.71s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/rds	788.368s
% make testacc TESTARGS='-run=TestAccRDSInstance_gp3' PKG=rds ACCTEST_PARALLELISM=3     
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_gp3 -timeout 180m
=== RUN   TestAccRDSInstance_gp3MySQL
=== PAUSE TestAccRDSInstance_gp3MySQL
=== RUN   TestAccRDSInstance_gp3Postgres
=== PAUSE TestAccRDSInstance_gp3Postgres
=== CONT  TestAccRDSInstance_gp3MySQL
=== CONT  TestAccRDSInstance_gp3Postgres
--- PASS: TestAccRDSInstance_gp3Postgres (693.72s)
--- PASS: TestAccRDSInstance_gp3MySQL (860.05s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/rds	865.392s
% make testacc TESTARGS='-run=TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops\|TestAccRDSInstance_basic\|TestAccRDSInstance_ReplicateSourceDB_iops\|TestAccRDSInstance_SnapshotIdentifier_io1Storage' PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops\|TestAccRDSInstance_basic\|TestAccRDSInstance_ReplicateSourceDB_iops\|TestAccRDSInstance_SnapshotIdentifier_io1Storage -timeout 180m
=== RUN   TestAccRDSInstance_basic
=== PAUSE TestAccRDSInstance_basic
=== RUN   TestAccRDSInstance_ReplicateSourceDB_iops
=== PAUSE TestAccRDSInstance_ReplicateSourceDB_iops
=== RUN   TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== PAUSE TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== RUN   TestAccRDSInstance_SnapshotIdentifier_io1Storage
=== PAUSE TestAccRDSInstance_SnapshotIdentifier_io1Storage
=== CONT  TestAccRDSInstance_basic
=== CONT  TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== CONT  TestAccRDSInstance_ReplicateSourceDB_iops
--- PASS: TestAccRDSInstance_basic (367.23s)
=== CONT  TestAccRDSInstance_SnapshotIdentifier_io1Storage
--- PASS: TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops (1467.13s)
--- PASS: TestAccRDSInstance_ReplicateSourceDB_iops (1548.51s)
--- PASS: TestAccRDSInstance_SnapshotIdentifier_io1Storage (1418.08s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/rds	1790.471s

Acceptance test output: % make testacc TESTARGS='-run=TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql' PKG=rds ACCTEST_PARALLELISM=3 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3 -run=TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql -timeout 180m === RUN TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql === PAUSE TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql === CONT TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql --- PASS: TestAccRDSInstance_EnabledCloudWatchLogsExports_postgresql (614.94s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/rds 619.983s

Acceptance test output: % make testacc TESTARGS='-run=TestAccRDSInstance_gp3' PKG=rds ACCTEST_PARALLELISM=3 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3 -run=TestAccRDSInstance_gp3 -timeout 180m === RUN TestAccRDSInstance_gp3MySQL === PAUSE TestAccRDSInstance_gp3MySQL === RUN TestAccRDSInstance_gp3Postgres === PAUSE TestAccRDSInstance_gp3Postgres === CONT TestAccRDSInstance_gp3MySQL === CONT TestAccRDSInstance_gp3Postgres --- PASS: TestAccRDSInstance_gp3Postgres (693.72s) --- PASS: TestAccRDSInstance_gp3MySQL (860.05s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/rds 865.392s

Acceptance test output: % make testacc TESTARGS='-run=TestAccRDSInstance_gp3SQLServer' PKG=rds ACCTEST_PARALLELISM=3 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3 -run=TestAccRDSInstance_gp3SQLServer -timeout 180m === RUN TestAccRDSInstance_gp3SQLServer === PAUSE TestAccRDSInstance_gp3SQLServer === CONT TestAccRDSInstance_gp3SQLServer --- PASS: TestAccRDSInstance_gp3SQLServer (1061.41s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/rds 1066.408s

Acceptance test output: % make testacc TESTARGS='-run=TestAccRDSInstance_storageTypePostgres' PKG=rds ACCTEST_PARALLELISM=3 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3 -run=TestAccRDSInstance_storageTypePostgres -timeout 180m === RUN TestAccRDSInstance_storageTypePostgres === PAUSE TestAccRDSInstance_storageTypePostgres === CONT TestAccRDSInstance_storageTypePostgres --- PASS: TestAccRDSInstance_storageTypePostgres (796.18s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/rds 801.356s

github-actions · 2022-12-07T22:15:15Z

Community Note

Voting for Prioritization

Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

jar-b

LGTM 🎉

+ make testacc TESTARGS=-run=TestAccRDSInstance_gp3SQLServer PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_gp3SQLServer -timeout 180m
=== RUN   TestAccRDSInstance_gp3SQLServer
=== PAUSE TestAccRDSInstance_gp3SQLServer
=== CONT  TestAccRDSInstance_gp3SQLServer
--- PASS: TestAccRDSInstance_gp3SQLServer (1095.72s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/rds        1098.790s
+ make testacc TESTARGS=-run=TestAccRDSInstance_storageThroughput PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_storageThroughput -timeout 180m
=== RUN   TestAccRDSInstance_storageThroughput
=== PAUSE TestAccRDSInstance_storageThroughput
=== CONT  TestAccRDSInstance_storageThroughput
--- PASS: TestAccRDSInstance_storageThroughput (732.48s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/rds        735.380s
+ make testacc TESTARGS=-run=TestAccRDSInstance_gp3 PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_gp3 -timeout 180m
=== RUN   TestAccRDSInstance_gp3MySQL
=== PAUSE TestAccRDSInstance_gp3MySQL
=== RUN   TestAccRDSInstance_gp3Postgres
=== PAUSE TestAccRDSInstance_gp3Postgres
=== RUN   TestAccRDSInstance_gp3SQLServer
=== PAUSE TestAccRDSInstance_gp3SQLServer
=== CONT  TestAccRDSInstance_gp3MySQL
=== CONT  TestAccRDSInstance_gp3SQLServer
=== CONT  TestAccRDSInstance_gp3Postgres
--- PASS: TestAccRDSInstance_gp3Postgres (817.02s)
--- PASS: TestAccRDSInstance_gp3MySQL (832.30s)
--- PASS: TestAccRDSInstance_gp3SQLServer (1062.37s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/rds        1065.288s
+ make testacc 'TESTARGS=-run=TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops\|TestAccRDSInstance_basic\|TestAccRDSInstance_ReplicateSourceDB_iops\|TestAccRDSInstance_SnapshotIdentifier_io1Storage' PKG=rds ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/rds/... -v -count 1 -parallel 3  -run=TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops\|TestAccRDSInstance_basic\|TestAccRDSInstance_ReplicateSourceDB_iops\|TestAccRDSInstance_SnapshotIdentifier_io1Storage -timeout 180m
=== RUN   TestAccRDSInstance_basic
=== PAUSE TestAccRDSInstance_basic
=== RUN   TestAccRDSInstance_ReplicateSourceDB_iops
=== PAUSE TestAccRDSInstance_ReplicateSourceDB_iops
=== RUN   TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== PAUSE TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== RUN   TestAccRDSInstance_SnapshotIdentifier_io1Storage
=== PAUSE TestAccRDSInstance_SnapshotIdentifier_io1Storage
=== CONT  TestAccRDSInstance_basic
=== CONT  TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops
=== CONT  TestAccRDSInstance_SnapshotIdentifier_io1Storage
--- PASS: TestAccRDSInstance_basic (530.24s)
=== CONT  TestAccRDSInstance_ReplicateSourceDB_iops
--- PASS: TestAccRDSInstance_SnapshotIdentifier_io1Storage (1301.87s)
--- PASS: TestAccRDSInstance_ReplicateSourceDB_allocatedStorageAndIops (1654.60s)
--- PASS: TestAccRDSInstance_ReplicateSourceDB_iops (1502.45s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/rds        2035.613s

github-actions · 2022-12-08T21:59:48Z

This functionality has been released in v4.46.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

etiennechabert · 2022-12-09T13:35:01Z

@ewbankkit thanks again for your effort, I was able to switch my RDS instance from gp2 to gp3 thanks to this PR.

But my pipelines are now blocked from further applies, if I don't specify iops, see: #28271

cdl-danielchapman · 2022-12-12T09:56:12Z

We have an issue where if we build an RDS instance at 100GiB specifying GP3 storage then the module dynamically sets IOPS to 3000 and throughput to 125MiB as expected

However, if we change the allocated_storage to 400GiB or more and apply that the apply fails with the below error

Error: updating RDS DB Instance (pg-prod-team-gp3): operation error RDS: ModifyDBInstance, https response error StatusCode: 400, RequestID: 970841cc-a412-4147-820c-700cb17e27fd, api error InvalidParameterCombination: Invalid iops value for engine name postgres and storage type gp3: 3000

If we build an RDS instance from a 100GiB snapshot but specify 400GiB in the terraform code at initial build it does the modification as expected and increases the storage/IOPS/throughput

But if we modify it after the instance has been built it errors. Ideally if it could dynamically lookup the values for IOPS/throughput that would be great. If they've been set in the code then set them to that but if they're null or not set to use the defaults i.e <400GiB 3K IOPS/125 MiB/S or >=400GiB 12K IOPS 500MiB/s

etiennechabert · 2022-12-12T13:13:42Z

@cdl-danielchapman maybe you want to give a 👍 to this issue I created: #28271

Sounds like exactly the problem I am having ?

I like your proposal to have the provider setting the values by default based on what is documented regarding gp3

cdl-danielchapman · 2022-12-12T14:00:04Z

@etiennechabert - done. Apologies I missed your earlier comment.

github-actions · 2023-01-12T02:16:22Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

ewbankkit added 4 commits December 7, 2022 13:49

github-actions bot added service/rds Issues and PRs that pertain to the rds service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. size/L Managed by automation to categorize the size of a PR. labels Dec 7, 2022

jar-b approved these changes Dec 8, 2022

View reviewed changes

ewbankkit merged commit 49f634b into main Dec 8, 2022

ewbankkit deleted the b-aws_db-instance-gp3.update branch December 8, 2022 16:59

github-actions bot added this to the v4.46.0 milestone Dec 8, 2022

ewbankkit mentioned this pull request Dec 8, 2022

Add CHANGELOG entry #28261

Merged

etiennechabert mentioned this pull request Dec 9, 2022

[Bug]: After switching an RDS instance from gp2 to gp3, following apply will fail if IOPS is not specified #28271

Closed

etiennechabert mentioned this pull request Dec 12, 2022

[Bug]: With gp3 storage_type for aws_db_instance cannot increase storage with gp3, or change to gp3, with storage < 400GB #28160

Closed

github-actions bot locked as resolved and limited conversation to collaborators Jan 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

r/aws_db_instance: Fix error modifying `allocated_storage` when `storage_type` is `"gp3"` #28243

r/aws_db_instance: Fix error modifying `allocated_storage` when `storage_type` is `"gp3"` #28243

ewbankkit commented Dec 7, 2022 •

edited

Loading

github-actions bot commented Dec 7, 2022

jar-b left a comment

github-actions bot commented Dec 8, 2022

etiennechabert commented Dec 9, 2022

cdl-danielchapman commented Dec 12, 2022

etiennechabert commented Dec 12, 2022

cdl-danielchapman commented Dec 12, 2022

github-actions bot commented Jan 12, 2023

r/aws_db_instance: Fix error modifying allocated_storage when storage_type is "gp3" #28243

r/aws_db_instance: Fix error modifying allocated_storage when storage_type is "gp3" #28243

Conversation

ewbankkit commented Dec 7, 2022 • edited Loading

Description

Relations

Output from Acceptance Testing

github-actions bot commented Dec 7, 2022

Community Note

jar-b left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 8, 2022

etiennechabert commented Dec 9, 2022

cdl-danielchapman commented Dec 12, 2022

etiennechabert commented Dec 12, 2022

cdl-danielchapman commented Dec 12, 2022

github-actions bot commented Jan 12, 2023

r/aws_db_instance: Fix error modifying `allocated_storage` when `storage_type` is `"gp3"` #28243

r/aws_db_instance: Fix error modifying `allocated_storage` when `storage_type` is `"gp3"` #28243

ewbankkit commented Dec 7, 2022 •

edited

Loading