Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing S3 bucket with inline policy creates unmanageable policy object #12805

Closed
alkalinecoffee opened this issue Apr 13, 2020 · 5 comments · Fixed by #14121
Closed

Importing S3 bucket with inline policy creates unmanageable policy object #12805

alkalinecoffee opened this issue Apr 13, 2020 · 5 comments · Fixed by #14121
Assignees
Labels
service/s3 Issues and PRs that pertain to the s3 service.
Milestone

Comments

@alkalinecoffee
Copy link
Contributor

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14
+ provider.aws v1.54.0
+ provider.template v1.0.0

Affected Resource(s)

  • aws_s3_bucket
  • aws_s3_bucket_policy

Terraform Configuration Files

resource "aws_s3_bucket" "bucket" {
  name = "my-bucket
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
  ...

Expected Behavior

I'm specifying an aws_s3_bucket object with an in-line policy value shown above and importing it into my state.

After importing, I make a change to the policy and run terraform plan. The plan should show only policy changes on my newly imported aws_s3_bucket object.

Actual Behavior

After importing, it appears that the policy is internally attached to an aws_s3_bucket_policy object, which does not exist in my configuration and is unmanageable without further config changes.

Because my policy is specified in-line with the aws_s3_bucket, the terraform plan shows that the policy must be created directly on the bucket, and the policy object be removed:

  ~ aws_s3_bucket.bucket
      policy:        "" => "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": ..."

  - aws_s3_bucket_policy.bucket

Plan: 0 to add, 1 to change, 1 to destroy.

This is confusing as according to the plan, the policy should be deleted, then re-attached directly to the S3 bucket. So we're hesitant about running this plan in production.

To me, if an S3 object with a policy is imported into configuration that specifies a policy in-line, that policy should be attached directly to the aws_s3_bucket object, and the aws_s3_bucket_policy object should not exist at all.

Steps to Reproduce

  1. Create bucket in S3 with a policy
  2. Import with terraform import aws_s3_bucket.bucket my-bucket
  3. terraform plan to see an unmanaged aws_s3_bucket_policy object
@ghost ghost added the service/s3 Issues and PRs that pertain to the s3 service. label Apr 13, 2020
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Apr 13, 2020
@bflad bflad removed the needs-triage Waiting for first response or review from a maintainer. label Apr 14, 2020
@bflad bflad added this to the v3.0.0 milestone Apr 14, 2020
@bflad
Copy link
Contributor

bflad commented Apr 14, 2020

Hi @alkalinecoffee 👋 Thank you for reporting this confusing behavior and sorry it exists to begin with.

We plan on removing any "complex" imports (where a single terraform import operation adds multiple resources into the Terraform state) across all resources in version 3.0.0 of the Terraform AWS Provider. (Since I notice Terraform 0.11 in your version information, please note that this major update of the Terraform AWS Provider will only support Terraform 0.12 and later.)

In the meantime (on any version of the Terraform CLI), you can execute the following to remove the "extra" resource from the Terraform state:

$ terraform state rm aws_s3_bucket_policy.bucket

This does not affect anything in the S3 API, only the Terraform state. 👍

@bflad bflad self-assigned this Jul 9, 2020
bflad added a commit that referenced this issue Jul 9, 2020
Reference: #394
Reference: #9001
Reference: #9508
Reference: #12805

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_acceleration (70.53s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (64.37s)
--- PASS: TestAccAWSS3Bucket_basic (37.90s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (39.08s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (32.28s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (39.25s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (68.80s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (67.23s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (37.19s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (44.32s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (37.21s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (38.50s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (37.77s)
--- PASS: TestAccAWSS3Bucket_generatedName (38.80s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (60.31s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (89.67s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (67.52s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (30.08s)
--- PASS: TestAccAWSS3Bucket_Logging (56.73s)
--- PASS: TestAccAWSS3Bucket_namePrefix (40.92s)
--- PASS: TestAccAWSS3Bucket_objectLock (68.34s)
--- PASS: TestAccAWSS3Bucket_Policy (97.07s)
--- PASS: TestAccAWSS3Bucket_region (34.45s)
--- PASS: TestAccAWSS3Bucket_Replication (159.22s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (94.18s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (95.79s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (28.62s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (167.50s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (55.52s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (58.02s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (67.28s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (19.65s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (119.32s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (171.42s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (65.51s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (92.38s)
--- PASS: TestAccAWSS3Bucket_Versioning (95.55s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (95.12s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (91.21s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (65.48s)
```
bflad added a commit that referenced this issue Jul 13, 2020
…14121)

Reference: #394
Reference: #9001
Reference: #9508
Reference: #12805

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_acceleration (70.53s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (64.37s)
--- PASS: TestAccAWSS3Bucket_basic (37.90s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (39.08s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (32.28s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (39.25s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (68.80s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (67.23s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (37.19s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (44.32s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (37.21s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (38.50s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (37.77s)
--- PASS: TestAccAWSS3Bucket_generatedName (38.80s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (60.31s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (89.67s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (67.52s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (30.08s)
--- PASS: TestAccAWSS3Bucket_Logging (56.73s)
--- PASS: TestAccAWSS3Bucket_namePrefix (40.92s)
--- PASS: TestAccAWSS3Bucket_objectLock (68.34s)
--- PASS: TestAccAWSS3Bucket_Policy (97.07s)
--- PASS: TestAccAWSS3Bucket_region (34.45s)
--- PASS: TestAccAWSS3Bucket_Replication (159.22s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (94.18s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (95.79s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (28.62s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (167.50s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (55.52s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (58.02s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (67.28s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (19.65s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (119.32s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (171.42s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (65.51s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (92.38s)
--- PASS: TestAccAWSS3Bucket_Versioning (95.55s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (95.12s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (91.21s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (65.48s)
```
@bflad
Copy link
Contributor

bflad commented Jul 13, 2020

The removal of the automatic aws_s3_bucket_policy resource import during aws_s3_bucket resource import has been merged and will release with version 3.0.0 of the Terraform AWS Provider, likely in two weeks. Please follow the v3.0.0 milestone for tracking the progress of that release. You can use the aws_s3_bucket_policy resource import support to import that resource directly after the provider upgrade, similar to all other Terraform resources that support import. 👍

@alkalinecoffee
Copy link
Contributor Author

Sounds good @bflad, thanks!

@ghost
Copy link

ghost commented Jul 31, 2020

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Aug 13, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/s3 Issues and PRs that pertain to the s3 service.
Projects
None yet
2 participants