Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aurora S3 import support #4366

Merged
merged 7 commits into from
Apr 27, 2018
Merged

Aurora S3 import support #4366

merged 7 commits into from
Apr 27, 2018

Conversation

jasonmcintosh
Copy link
Contributor

Similar to the RDS S3 XtraBackup support. Acceptance tests have passed though aurora takes forever and a day to run them.

@ghost ghost added the size/L Managed by automation to categorize the size of a PR. label Apr 26, 2018
@jasonmcintosh
Copy link
Contributor Author

Let me know if I've missed anything @bflad :) Should be nearly identical to the one for a db instance.

Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple copypasta fix below -- running the tests now 👍

@@ -432,6 +471,97 @@ func resourceAwsRDSClusterCreate(d *schema.ResourceData, meta interface{}) error

log.Printf("[DEBUG]: RDS Cluster create response: %s", resp)

} else if v, ok := d.GetOk("s3_import"); ok {
if _, ok := d.GetOk("master_password"); !ok {
return fmt.Errorf(`provider.aws: aws_db_instance: %s: "password": required field is not set`, d.Get("name").(string))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"password" should be "master_password" 😄 Same with username one below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed :) I've got an issue out on moving db_instance username to be master_username precisely b/c of some of this confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, for us, Aurora tests seem to be taking about 30 minutes. NOT ideal, but I don't see a way around it yet... trying a few things to reduce test time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use a special wrapper to parallelize the acceptance testing in our CI environment: https://github.com/jen20/teamcity-go-test/

It'd probably be nice if we offered something similar for local testing.

@bflad bflad added enhancement Requests to existing resources that expand the functionality or scope. service/rds Issues and PRs that pertain to the rds service. labels Apr 26, 2018
@ghost ghost added the size/L Managed by automation to categorize the size of a PR. label Apr 26, 2018
if isAWSErr(err, "InvalidParameterValue", "S3 bucket cannot be found") {
return resource.RetryableError(err)
}
return resource.NonRetryableError(err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we have another case to retry on 😄

--- FAIL: TestAccAWSRDSCluster_s3Restore (21.12s)
    testing.go:518: Step 0 error: Error applying: 1 error(s) occurred:
        
        * aws_rds_cluster.test: 1 error(s) occurred:
        
        * aws_rds_cluster.test: InvalidParameterValue: Files from the specified Amazon S3 bucket cannot be downloaded. Make sure that you have created an AWS Identity and Access Management (IAM) role that lets Amazon RDS access Amazon S3 for you.
            status code: 400, request id: 06f2db02-6d04-448a-a659-1e1550a92fa8
if isAWSErr(err, "InvalidParameterValue", "Files from the specified Amazon S3 bucket cannot be downloaded") {
  return resource.RetryableError(err)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... can add that, though this SEEMS like an AWS issue. We had no issues running here... will add this, but dang that seems like a really strange issue. Possibly a cross region issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah just found it in API docs. Bucket has to be created in same region as the RDS cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hoping the latest commit fixes this issue - sets the bucket region explicitly as part of the test.

@bflad bflad added this to the v1.17.0 milestone Apr 27, 2018
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @jasonmcintosh!! 🚀

25 tests passed (all tests)
=== RUN   TestAccAWSRDSCluster_missingUserNameCausesError
--- PASS: TestAccAWSRDSCluster_missingUserNameCausesError (4.07s)
=== RUN   TestAccAWSRDSCluster_basic
--- PASS: TestAccAWSRDSCluster_basic (100.09s)
=== RUN   TestAccAWSRDSCluster_importBasic
--- PASS: TestAccAWSRDSCluster_importBasic (100.51s)
=== RUN   TestAccAWSRDSCluster_encrypted
--- PASS: TestAccAWSRDSCluster_encrypted (111.38s)
=== RUN   TestAccAWSRDSCluster_updateIamRoles
--- PASS: TestAccAWSRDSCluster_updateIamRoles (117.06s)
=== RUN   TestAccAWSRDSCluster_updateTags
--- PASS: TestAccAWSRDSCluster_updateTags (124.91s)
=== RUN   TestAccAWSRDSCluster_kmsKey
--- PASS: TestAccAWSRDSCluster_kmsKey (138.75s)
=== RUN   TestAccAWSRDSCluster_generatedName
--- PASS: TestAccAWSRDSCluster_generatedName (140.88s)
=== RUN   TestAccAWSRDSCluster_namePrefix
--- PASS: TestAccAWSRDSCluster_namePrefix (143.06s)
=== RUN   TestAccAWSRDSCluster_takeFinalSnapshot
--- PASS: TestAccAWSRDSCluster_takeFinalSnapshot (160.84s)
=== RUN   TestAccAWSRDSCluster_iamAuth
--- PASS: TestAccAWSRDSCluster_iamAuth (107.80s)
=== RUN   TestAccAWSRDSCluster_EngineVersion
--- PASS: TestAccAWSRDSCluster_EngineVersion (109.97s)
=== RUN   TestAccAWSRDSCluster_backupsUpdate
--- PASS: TestAccAWSRDSCluster_backupsUpdate (122.75s)
=== RUN   TestAccAWSRDSCluster_Port
--- PASS: TestAccAWSRDSCluster_Port (255.87s)
=== RUN   TestAccAWSRDSClusterInstance_generatedName
--- PASS: TestAccAWSRDSClusterInstance_generatedName (628.49s)
=== RUN   TestAccAWSRDSClusterInstance_importBasic
--- PASS: TestAccAWSRDSClusterInstance_importBasic (666.79s)
=== RUN   TestAccAWSRDSClusterInstance_withInstancePerformanceInsights
--- PASS: TestAccAWSRDSClusterInstance_withInstancePerformanceInsights (686.18s)
=== RUN   TestAccAWSRDSClusterInstance_az
--- PASS: TestAccAWSRDSClusterInstance_az (687.83s)
=== RUN   TestAccAWSRDSClusterInstance_kmsKey
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (706.38s)
=== RUN   TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor
--- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (709.50s)
=== RUN   TestAccAWSRDSClusterInstance_namePrefix
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (722.68s)
=== RUN   TestAccAWSRDSClusterInstance_disappears
--- PASS: TestAccAWSRDSClusterInstance_disappears (732.38s)
=== RUN   TestAccAWSRDSCluster_s3Restore
--- PASS: TestAccAWSRDSCluster_s3Restore (1447.64s)
=== RUN   TestAccAWSRDSClusterInstance_basic
--- PASS: TestAccAWSRDSClusterInstance_basic (1672.04s)
=== RUN   TestAccAWSRDSCluster_EncryptedCrossRegionReplication
--- PASS: TestAccAWSRDSCluster_EncryptedCrossRegionReplication (2109.41s)

@bflad bflad merged commit 973f180 into hashicorp:master Apr 27, 2018
bflad added a commit that referenced this pull request Apr 27, 2018
@bflad
Copy link
Contributor

bflad commented May 2, 2018

This has been released in version 1.17.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ghost
Copy link

ghost commented Apr 6, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement Requests to existing resources that expand the functionality or scope. service/rds Issues and PRs that pertain to the rds service. size/L Managed by automation to categorize the size of a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants