Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

service/s3: Passthrough provider s3_force_path_style configuration for s3manager.GetBucketRegion calls #14481

Merged
merged 1 commit into from
Aug 6, 2020

Conversation

bflad
Copy link
Contributor

@bflad bflad commented Aug 5, 2020

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #14427

Release note for CHANGELOG:

* data-source/aws_s3_bucket: Ensure provider `s3_force_path_style` configuration is passed through for getting S3 Bucket location with non-AWS implementations
* resource/aws_s3_bucket: Ensure provider `s3_force_path_style` configuration is passed through for getting S3 Bucket location with non-AWS implementations

Previously on or after 3.0.0:

terraform {
  required_providers {
    aws = "3.0.0"
  }

  required_version = "0.12.29"
}

provider "aws" {
  region = "eu-west-1"

  access_key = "fakeKey"
  secret_key = "fakeKey"

  s3_force_path_style         = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    s3 = "http://localhost:9090"
  }
}

resource "aws_s3_bucket" "test" {
  bucket = "test-bucket"
}
$ docker run -p 9090:9090 -t adobe/s3mock
$ terraform apply
...
aws_s3_bucket.test: Creating...
aws_s3_bucket.test: Still creating... [10s elapsed]
aws_s3_bucket.test: Still creating... [20s elapsed]

Error: error getting S3 Bucket location: RequestError: send request failed
caused by: dial tcp: lookup test-bucket.localhost on 192.168.1.1:53: no such host

  on main.tf line 25, in resource "aws_s3_bucket" "test":
  25: resource "aws_s3_bucket" "test" {

Now:

$ terraform apply
...
aws_s3_bucket.test: Creating...
aws_s3_bucket.test: Creation complete after 0s [id=test-bucket]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Output from acceptance testing:

--- PASS: TestAccAWSS3Bucket_acceleration (53.30s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (51.07s)
--- PASS: TestAccAWSS3Bucket_basic (31.56s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (31.65s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (25.97s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (33.67s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (55.10s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (56.01s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (31.69s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (38.29s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (32.49s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (29.00s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (31.61s)
--- PASS: TestAccAWSS3Bucket_generatedName (30.69s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (48.35s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (79.24s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (52.70s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (25.59s)
--- PASS: TestAccAWSS3Bucket_Logging (50.13s)
--- PASS: TestAccAWSS3Bucket_namePrefix (30.79s)
--- PASS: TestAccAWSS3Bucket_objectLock (54.26s)
--- PASS: TestAccAWSS3Bucket_Policy (73.69s)
--- PASS: TestAccAWSS3Bucket_Replication (111.61s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (66.43s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (64.58s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (19.21s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (114.21s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (38.67s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (39.78s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (56.71s)
--- PASS: TestAccAWSS3Bucket_SameRegionReplicationSchemaV2 (51.20s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (12.38s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (99.87s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (148.81s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (50.58s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (80.43s)
--- PASS: TestAccAWSS3Bucket_Versioning (77.33s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (79.51s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (79.03s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (55.56s)

--- PASS: TestAccDataSourceS3Bucket_basic (28.96s)
--- PASS: TestAccDataSourceS3Bucket_website (29.28s)

…r s3manager.GetBucketRegion calls

Reference: #14427

Previously on or after 3.0.0:

```terraform
terraform {
  required_providers {
    aws = "3.0.0"
  }

  required_version = "0.12.29"
}

provider "aws" {
  region = "eu-west-1"

  access_key = "fakeKey"
  secret_key = "fakeKey"

  s3_force_path_style         = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    s3 = "http://localhost:9090"
  }
}

resource "aws_s3_bucket" "test" {
  bucket = "test-bucket"
}
```

```console
$ docker run -p 9090:9090 -t adobe/s3mock
$ terraform apply
...
aws_s3_bucket.test: Creating...
aws_s3_bucket.test: Still creating... [10s elapsed]
aws_s3_bucket.test: Still creating... [20s elapsed]

Error: error getting S3 Bucket location: RequestError: send request failed
caused by: dial tcp: lookup test-bucket.localhost on 192.168.1.1:53: no such host

  on main.tf line 25, in resource "aws_s3_bucket" "test":
  25: resource "aws_s3_bucket" "test" {
```

Now:

```console
$ terraform apply
...
aws_s3_bucket.test: Creating...
aws_s3_bucket.test: Creation complete after 0s [id=test-bucket]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
```

Output from acceptance testing:

```
--- PASS: TestAccAWSS3Bucket_acceleration (53.30s)
--- PASS: TestAccAWSS3Bucket_AclToGrant (51.07s)
--- PASS: TestAccAWSS3Bucket_basic (31.56s)
--- PASS: TestAccAWSS3Bucket_Bucket_EmptyString (31.65s)
--- PASS: TestAccAWSS3Bucket_Cors_Delete (25.97s)
--- PASS: TestAccAWSS3Bucket_Cors_EmptyOrigin (33.67s)
--- PASS: TestAccAWSS3Bucket_Cors_Update (55.10s)
--- PASS: TestAccAWSS3Bucket_disableDefaultEncryption_whenDefaultEncryptionIsEnabled (56.01s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenAES256IsUsed (31.69s)
--- PASS: TestAccAWSS3Bucket_enableDefaultEncryption_whenTypical (38.29s)
--- PASS: TestAccAWSS3Bucket_forceDestroy (32.49s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithEmptyPrefixes (29.00s)
--- PASS: TestAccAWSS3Bucket_forceDestroyWithObjectLockEnabled (31.61s)
--- PASS: TestAccAWSS3Bucket_generatedName (30.69s)
--- PASS: TestAccAWSS3Bucket_GrantToAcl (48.35s)
--- PASS: TestAccAWSS3Bucket_LifecycleBasic (79.24s)
--- PASS: TestAccAWSS3Bucket_LifecycleExpireMarkerOnly (52.70s)
--- PASS: TestAccAWSS3Bucket_LifecycleRule_Expiration_EmptyConfigurationBlock (25.59s)
--- PASS: TestAccAWSS3Bucket_Logging (50.13s)
--- PASS: TestAccAWSS3Bucket_namePrefix (30.79s)
--- PASS: TestAccAWSS3Bucket_objectLock (54.26s)
--- PASS: TestAccAWSS3Bucket_Policy (73.69s)
--- PASS: TestAccAWSS3Bucket_Replication (111.61s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (66.43s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (64.58s)
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (19.21s)
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (114.21s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (38.67s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (39.78s)
--- PASS: TestAccAWSS3Bucket_RequestPayer (56.71s)
--- PASS: TestAccAWSS3Bucket_SameRegionReplicationSchemaV2 (51.20s)
--- PASS: TestAccAWSS3Bucket_shouldFailNotFound (12.38s)
--- PASS: TestAccAWSS3Bucket_tagsWithNoSystemTags (99.87s)
--- PASS: TestAccAWSS3Bucket_tagsWithSystemTags (148.81s)
--- PASS: TestAccAWSS3Bucket_UpdateAcl (50.58s)
--- PASS: TestAccAWSS3Bucket_UpdateGrant (80.43s)
--- PASS: TestAccAWSS3Bucket_Versioning (77.33s)
--- PASS: TestAccAWSS3Bucket_Website_Simple (79.51s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (79.03s)
--- PASS: TestAccAWSS3Bucket_WebsiteRoutingRules (55.56s)

--- PASS: TestAccDataSourceS3Bucket_basic (28.96s)
--- PASS: TestAccDataSourceS3Bucket_website (29.28s)
```
@bflad bflad added bug Addresses a defect in current functionality. service/s3 Issues and PRs that pertain to the s3 service. labels Aug 5, 2020
@bflad bflad requested a review from a team August 5, 2020 20:22
@ghost ghost added size/XS Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Aug 5, 2020
@anGie44 anGie44 self-requested a review August 6, 2020 15:56
Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 Confirmed update in behavior using local s3mock

@bflad bflad added this to the v3.1.0 milestone Aug 6, 2020
@bflad bflad merged commit 8d0c8b6 into master Aug 6, 2020
@bflad bflad deleted the b-aws_s3_bucket-GetBucketRegion-s3_force_path_style branch August 6, 2020 17:01
bflad added a commit that referenced this pull request Aug 6, 2020
@ghost
Copy link

ghost commented Aug 7, 2020

This has been released in version 3.1.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Sep 5, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Sep 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/s3 Issues and PRs that pertain to the s3 service. size/XS Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to create S3 buckets on local mocks since 3.0.0
2 participants