Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

resource/aws_s3_bucket: Prevent empty replication_configuration rules filter crash #7887

Merged
merged 3 commits into from
Mar 14, 2019
Merged

resource/aws_s3_bucket: Prevent empty replication_configuration rules filter crash #7887

merged 3 commits into from
Mar 14, 2019

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Mar 11, 2019

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Closes #7427
Closes #6600

Changes proposed in this pull request (see commit messages for full details):

  • resource/aws_s3_bucket: Prevent empty replication_configuration rules filter crash. This is a best effort fix given this may be caused by a manual console update and we do not have a replicating Terraform configuration. The previous code was missing a nil check before type assertion.
  • tests/resource/aws_s3_bucket: Prevent CheckDestroy errors due to eventual consistency. Previously, errors could be returned since bucket deletion may take a few seconds to propagate.
  • tests/service/s3: Add mybucket and mylogs prefixes to sweeping. These two test bucket name prefixes were piling up. We are still keeping the prefix based checks with S3 Bucket names to allow internal users to migrate off their existing buckets.

Output from acceptance testing:

--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (17.44s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (29.29s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (29.94s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (31.64s)
--- PASS: TestAccAWSS3Bucket_importBasic (33.49s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (41.70s)
--- PASS: TestAccAWSS3Bucket_Logging (47.80s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (52.04s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (52.96s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (53.36s)
--- PASS: TestAccAWSS3Bucket_objectLock (53.81s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (58.86s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (48.75s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (70.75s)
--- PASS: TestAccAWSS3Bucket_Policy (70.96s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (72.03s)
--- PASS: TestAccAWSS3Bucket_Versioning (72.13s)
--- PASS: TestAccAWSS3Bucket_Lifecycle (73.01s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (47.45s)
--- PASS: TestAccAWSS3Bucket_basic (24.91s)
--- PASS: TestAccAWSS3Bucket_namePrefix (24.41s)
--- PASS: TestAccAWSS3Bucket_region (31.70s)
--- PASS: TestAccAWSS3Bucket_generatedName (26.78s)
--- PASS: TestAccAWSS3Bucket_acceleration (49.96s)
--- PASS: TestAccAWSS3Bucket_importWithPolicy (35.90s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (48.55s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (70.34s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (70.38s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (148.13s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (216.56s)
--- PASS: TestAccAWSS3Bucket_Replication (225.08s)

@bflad
tests/resource/aws_s3_bucket: Prevent CheckDestroy errors due to even…
2e36fca 
…tual consistency

Previously, these errors could be returned since bucket deletion may take a few seconds to propagate:

```
    testing.go:599: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.

        Error: Check failed: AWS S3 Bucket still exists: tf-test-bucket-865512847222774644
```

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_basic (26.11s)
```
@bflad
tests/service/s3: Add mybucket and mylogs prefixes to sweeping
30e646b 
These two test bucket name prefixes were piling up. We are still keeping the prefix based checks with S3 Bucket names to allow internal users to migrate off their existing buckets.

Previous output from acceptance testing:

```
    testing.go:538: Step 0 error: Error applying: 1 error occurred:
        	* aws_s3_bucket.destination: 1 error occurred:
        	* aws_s3_bucket.destination: Error creating S3 bucket: TooManyBuckets: You have attempted to create more buckets than allowed
```

Output from test sweeper:

```
$ go test ./aws -v -sweep=us-west-2 -sweep-run=aws_s3_bucket -timeout 10h
...
2019/03/11 08:50:53 [INFO] Deleting S3 Bucket: mybucket.1023155772881419661
2019/03/11 08:50:54 [INFO] Deleting S3 Bucket: mybucket.1577116853589958138
2019/03/11 08:50:54 [INFO] Deleting S3 Bucket: mybucket.1626923880857291096
2019/03/11 08:50:55 [INFO] Deleting S3 Bucket: mybucket.1918546658572021590
2019/03/11 08:50:56 [INFO] Deleting S3 Bucket: mybucket.2529084512248573413
2019/03/11 08:50:57 [INFO] Deleting S3 Bucket: mybucket.3093679243342824574
2019/03/11 08:50:57 [INFO] Deleting S3 Bucket: mybucket.3585723648527080440
2019/03/11 08:50:58 [INFO] Deleting S3 Bucket: mybucket.3947240404810737197
2019/03/11 08:50:59 [INFO] Deleting S3 Bucket: mybucket.4249057747579529260
2019/03/11 08:51:00 [INFO] Deleting S3 Bucket: mybucket.4313397459697999834
2019/03/11 08:51:00 [INFO] Deleting S3 Bucket: mybucket.4525762606472232559
2019/03/11 08:51:01 [INFO] Deleting S3 Bucket: mybucket.4648730065658883877
2019/03/11 08:51:01 [INFO] Deleting S3 Bucket: mybucket.4823172894294433332
2019/03/11 08:51:03 [INFO] Deleting S3 Bucket: mybucket.5586690765808533355
2019/03/11 08:51:04 [INFO] Deleting S3 Bucket: mybucket.5862186744102500925
2019/03/11 08:51:05 [INFO] Deleting S3 Bucket: mybucket.6558243541687708420
2019/03/11 08:51:05 [INFO] Deleting S3 Bucket: mybucket.8348088465902968848
2019/03/11 08:51:06 [INFO] Deleting S3 Bucket: mybucket.8790461889364761310
2019/03/11 08:51:07 [INFO] Deleting S3 Bucket: mybucket.942914357360372416
2019/03/11 08:51:08 [INFO] Deleting S3 Bucket: mylogs.1023155772881419661
2019/03/11 08:51:09 [INFO] Deleting S3 Bucket: mylogs.1577116853589958138
2019/03/11 08:51:09 [INFO] Deleting S3 Bucket: mylogs.1626923880857291096
2019/03/11 08:51:10 [INFO] Deleting S3 Bucket: mylogs.1918546658572021590
2019/03/11 08:51:11 [INFO] Deleting S3 Bucket: mylogs.2831805700791414627
2019/03/11 08:51:13 [INFO] Deleting S3 Bucket: mylogs.3093679243342824574
2019/03/11 08:51:14 [INFO] Deleting S3 Bucket: mylogs.3102934574929067207
2019/03/11 08:51:15 [INFO] Deleting S3 Bucket: mylogs.320626190810004195
2019/03/11 08:51:17 [INFO] Deleting S3 Bucket: mylogs.3770914409175143465
2019/03/11 08:51:17 [INFO] Deleting S3 Bucket: mylogs.3947240404810737197
2019/03/11 08:51:18 [INFO] Deleting S3 Bucket: mylogs.4823172894294433332
2019/03/11 08:51:19 [INFO] Deleting S3 Bucket: mylogs.5107202730033269573
2019/03/11 08:51:19 [INFO] Deleting S3 Bucket: mylogs.5586690765808533355
2019/03/11 08:51:20 [INFO] Deleting S3 Bucket: mylogs.5788936017557252310
2019/03/11 08:51:21 [INFO] Deleting S3 Bucket: mylogs.5862186744102500925
2019/03/11 08:51:21 [INFO] Deleting S3 Bucket: mylogs.6558243541687708420
2019/03/11 08:51:22 [INFO] Deleting S3 Bucket: mylogs.7781653724497201590
2019/03/11 08:51:23 [INFO] Deleting S3 Bucket: mylogs.8348088465902968848
2019/03/11 08:51:24 [INFO] Deleting S3 Bucket: mylogs.8790461889364761310
2019/03/11 08:51:24 [INFO] Deleting S3 Bucket: mylogs.942914357360372416
...
2019/03/11 08:51:25 Sweeper Tests ran:
	- aws_s3_bucket_object
	- aws_s3_bucket
ok  	github.com/terraform-providers/terraform-provider-aws/aws	71.008s
```
@bflad
resource/aws_s3_bucket: Prevent panic with empty replication_configur…
e204667 
…ation rules filter

References:

* #7427
* #6600

This is a best effort fix given this may be caused by a manual console update and we do not have a replicating Terraform configuration. The previous code was missing a `nil` check before type assertion.

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (17.44s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (29.29s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (29.94s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (31.64s)
--- PASS: TestAccAWSS3Bucket_importBasic (33.49s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (41.70s)
--- PASS: TestAccAWSS3Bucket_Logging (47.80s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (52.04s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (52.96s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (53.36s)
--- PASS: TestAccAWSS3Bucket_objectLock (53.81s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (58.86s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (48.75s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (70.75s)
--- PASS: TestAccAWSS3Bucket_Policy (70.96s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (72.03s)
--- PASS: TestAccAWSS3Bucket_Versioning (72.13s)
--- PASS: TestAccAWSS3Bucket_Lifecycle (73.01s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (47.45s)
--- PASS: TestAccAWSS3Bucket_basic (24.91s)
--- PASS: TestAccAWSS3Bucket_namePrefix (24.41s)
--- PASS: TestAccAWSS3Bucket_region (31.70s)
--- PASS: TestAccAWSS3Bucket_generatedName (26.78s)
--- PASS: TestAccAWSS3Bucket_acceleration (49.96s)
--- PASS: TestAccAWSS3Bucket_importWithPolicy (35.90s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (48.55s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (70.34s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (70.38s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (148.13s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (216.56s)
--- PASS: TestAccAWSS3Bucket_Replication (225.08s)
```
@bflad bflad added bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. service/s3 Issues and PRs that pertain to the s3 service. labels Mar 11, 2019
@bflad bflad requested a review from a team March 11, 2019 13:14
@ghost ghost added size/S Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Mar 11, 2019
This was referenced Mar 11, 2019
Closed
Closed
nywilken
nywilken approved these changes Mar 12, 2019
View reviewed changes
Copy link
Contributor

@nywilken nywilken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bflad bflad added this to the v2.2.0 milestone Mar 14, 2019
@bflad bflad merged commit 1281fae into master Mar 14, 2019
@bflad bflad deleted the b-aws_s3_bucket-prevent-empty-filter-crash branch March 14, 2019 16:42
bflad added a commit that referenced this pull request Mar 14, 2019
@bflad
Update CHANGELOG for #7887
0b4f3b4
@bflad
Copy link
Contributor Author

bflad commented Mar 15, 2019

This has been released in version 2.2.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ghost
Copy link

ghost commented Mar 31, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Mar 31, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@nywilken nywilken nywilken approved these changes

Assignees
No one assigned
Labels
bug Addresses a defect in current functionality. crash Results from or addresses a Terraform crash or kernel panic. service/s3 Issues and PRs that pertain to the s3 service. size/S Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v2.2.0
Development

Successfully merging this pull request may close these issues.

Terraform Crash during Terraform apply Terraform crash while changing s3 bucket
2 participants
@bflad @nywilken