Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix error creating aws_db_instance from snapshot where snapshot uses storage_type io1 #5800

Merged
merged 2 commits into from
Feb 22, 2019

Conversation

dannosaur
Copy link

There's no issue number for this, this is something I've stumbled across when trying to build an environment in a very specific way.

When restoring a DB instance from a snapshot, where the snapshot uses io1 storage type, the initial create succeeds, but the second step of the create within terraform that modifies the DB instance to sync it up with the resource state fails due to the fact that the iops config item isn't passed. The error it crashes out with is:

* aws_db_instance.db: error modifying DB Instance (terraform-2018090607083682360000000e): InvalidParameterCombination: Both storage size and iops must be specified when modifying storage size or iops on an DB Instance that has iops

Changes proposed in this pull request:

  • Move the iops config element from the initial create to the secondary "modify" stage of the create (the initial iops value gets inherited from the snapshot, so this isn't required in the initial stage

Output from acceptance testing: N/A

@bflad bflad added bug Addresses a defect in current functionality. service/rds Issues and PRs that pertain to the rds service. labels Sep 6, 2018
@bflad
Copy link
Contributor

bflad commented Sep 6, 2018

Hi @dannosaur 👋 Thanks for submitting this!

Can you please provide an acceptance test or an example of your configuration so we can verify this as well as prevent future regressions?

@bflad bflad added the waiting-response Maintainers are waiting on response from community or contributor. label Sep 6, 2018
@dannosaur
Copy link
Author

I'm actually unsure as to how one might build an acceptance test for this. The issue being that in order to hit this piece of code, one must restore a database from a snapshot. As I understand it (please excuse the fact that I have extremely little knowledge of Go), these tests work by creating real resources and testing that the result is as expected? If this is the case, then whoever's testing has a prerequisite in that they need a snapshot ARN to hand in order to restore it from. Public snapshots are a thing, but I can't find a list of anything to use (and the removal of said snapshot would break the test, obviously not what we want).

How would you actually go about this? Is there another method where you can "fake" the calls somehow and test that the calls made are what's expected and in the right order?

Help appreciated on this one. I'd love to contribute more.

@bflad
Copy link
Contributor

bflad commented Sep 6, 2018

@dannosaur no worries -- we have existing acceptance testing similar to what needs to be done here where we create a database, snapshot it, then create a second database that is restored from snapshot. Check out:

https://github.com/terraform-providers/terraform-provider-aws/blob/2bcb2446484e2f9a8b98af37918f2b4c21698cb8/aws/resource_aws_db_instance_test.go#L709-L734

and

https://github.com/terraform-providers/terraform-provider-aws/blob/2bcb2446484e2f9a8b98af37918f2b4c21698cb8/aws/resource_aws_db_instance_test.go#L3554-L3579

Then to run it:

export AWS_PROFILE=... # or export AWS_ACCESS_KEY_ID=... + export AWS_SECRET_ACCESS_KEY=...
make testacc TEST=./aws TESTARGS='-run=TestAccAWSDBInstance_SnapshotIdentifier_AllocatedStorage'

@DI-DaveGoodine
Copy link

I am seeing this issue as well in:
provider.aws: version = "~> 1.46"

I am trying to create a postgresql db instance from a snapshot made on an io1 db to a new io1 db.

@ghost ghost removed the waiting-response Maintainers are waiting on response from community or contributor. label Nov 20, 2018
@bvargheseamplify
Copy link

Our RDS sandbox builds are failing due to this. Any ETA on the release of the fix?

@hylaride
Copy link

We've been getting hit with this bug for a few months now. Current setup:

Terraform v0.11.10
+ provider.aws v1.51.0
+ provider.null v1.0.0
+ provider.template v1.0.0

@sklarsa
Copy link

sklarsa commented Jan 31, 2019

Is this not being merged because of a lack of tests? If so, I can take on this task based on @bflad's comments from above. But if someone else is working on it, I don't want to duplicate effort...

This is pretty critical for my company's workflow, so it would be great to see it fixed!

@dannosaur
Copy link
Author

@sklarsa yes, that's correct. I simply lack the time to sit and study how the test suite works, and learn enough of Go's syntax to write correct and valid tests that pass.

@sklarsa
Copy link

sklarsa commented Jan 31, 2019 via email

@sgript
Copy link

sgript commented Feb 9, 2019

Any updates? Running into the same issue - for the same use-case, whereby we're trying to create an RDS Postgres 10 DB (io1) from a RDS Postgres 10 DB (io1) snapshot, getting the error:

 aws_db_instance.database_resource: error modifying DB Instance (terraform-rentals-staging): InvalidParameterCombination: Both storage size and iops must be specified when modifying storage size or iops on an DB Instance that has iops

Initial creation succeeds, modification fails as described in the symptoms.


Any workarounds if no fixes? As this is blocking our company from re-creating existing (populated) databases in Terraform.

@sklarsa
Copy link

sklarsa commented Feb 11, 2019

My PR #7426 addresses this problem, but it needs review

@hernan82arg
Copy link

hitting same issue here... can we please merge? thanks

@sgript
Copy link

sgript commented Feb 12, 2019

Would be amazing. This is the only thing seemingly stopping us from terraforming our database with an existing snapshot.

@sgript
Copy link

sgript commented Feb 12, 2019

cc @bflad

@bflad bflad added this to the v1.60.0 milestone Feb 22, 2019
@bflad
Copy link
Contributor

bflad commented Feb 22, 2019

As noted here: #7426 (review)

Merging this first so @dannosaur gets proper attribution then merging #7426. Thanks so much @dannosaur and @sklarsa

@bflad bflad merged commit 02995bc into hashicorp:master Feb 22, 2019
bflad added a commit that referenced this pull request Feb 22, 2019
@bflad
Copy link
Contributor

bflad commented Feb 22, 2019

This has been released in version 1.60.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ghost
Copy link

ghost commented Mar 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Mar 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/rds Issues and PRs that pertain to the rds service.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants