[Bug]: Migrate aws_s3_object from provider 4 to 5 is not working at first apply #31633

quentin9696 · 2023-05-29T22:21:12Z

Terraform Core Version

1.4.6

AWS Provider Version

5.0.1, 4.67.0

Affected Resource(s)

aws_s3_object

Expected Behavior

Nothing to do to upgrade from provider 4 to 5

Actual Behavior

Bump the provider version from 4 to 5 trigger a change in the plan:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_object.this will be updated in-place
  ~ resource "aws_s3_object" "this" {
      - acl                    = "private" -> null
        id                     = "foo"
        tags                   = {}
        # (12 unchanged attributes hidden)
    }

And the apply trigger an error:

│ Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
│ 	status code: 400, request id: QWQSKETKTX6TYF1X, host id: oL9WH0HNofYSqyI0XuMi2wpFJsLmHU3kR05QGt0MUpxHW6OmInyCQy4OU0ocC75Tq7mk+ELV5L0pmWO8u6eoM4Kat5JYtIOumXwNGw3Qd2s=
│ 
│   with aws_s3_object.this,
│   on main.tf line 18, in resource "aws_s3_object" "this":
│   18: resource "aws_s3_object" "this" {

Relevant Error/Panic Output Snippet

│ Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
│ 	status code: 400, request id: XXXXXXX, host id: XXXXXXXXXXXXXX
│ 
│   with aws_s3_object.this,
│   on main.tf line 18, in resource "aws_s3_object" "this":
│   18: resource "aws_s3_object" "this" {


Apply a 2nd time will cause an `No changes. Your infrastructure matches the configuration.`

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.0, < 5"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "example" {
  bucket = "test-quentin9696"
}

resource "aws_s3_object" "this" {
  bucket = aws_s3_bucket.example.id
  key    = "foo"

  content_base64 = base64encode("Hello World")
  source_hash    = md5(base64encode("Hello World"))
}

Steps to Reproduce

Using configuration above, terraform init
terraform apply
Change the provider version to pin the version 5 (update version = ">= 4.0, < 5" to version = "> 5")
terraform init -upgrade
terraform apply
Get the error
terraform apply
Get the message No changes. Your infrastructure matches the configuration.

Debug Output

No response

Panic Output

No response

Important Factoids

I think the provider should upgrade the state internally instead of waiting for the double update

References

No response

Would you like to implement a fix?

No

The text was updated successfully, but these errors were encountered:

github-actions · 2023-05-29T22:21:23Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

justinretzolk · 2023-06-01T20:53:31Z

Hey @quentin9696 👋 Thank you for taking the time to raise this! So that we have the information necessary to look into this, can you supply debug logs (redacted as needed) as well?

pguinard-public-com · 2023-06-02T14:50:12Z

Here's the logs from a 5.0.1 provider which generate the error for me.

terraform plan:

2023-06-02T08:37:13.205-0600 [DEBUG] ProviderTransformer: "aws_s3_object.configurationfolders" (*terraform.NodeValidatableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-06-02T08:37:14.480-0600 [DEBUG] ReferenceTransformer: "aws_s3_object.configurationfolders" references: [local.all_services (expand) local.all_services (expand) var.environment aws_s3_bucket.xxx]
2023-06-02T08:37:21.185-0600 [DEBUG] ProviderTransformer: "aws_s3_object.configurationfolders (expand)" (*terraform.nodeExpandPlannableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-06-02T08:37:22.513-0600 [DEBUG] ReferenceTransformer: "aws_s3_object.configurationfolders (expand)" references: [local.all_services (expand) aws_s3_bucket.xxx (expand) local.all_services (expand) var.environment]
2023-06-02T08:37:27.280-0600 [DEBUG] ReferenceTransformer: "aws_s3_object.configurationfolders[0]" references: []
2023-06-02T08:37:27.482-0600 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for 
2023-06-02T08:35:43.649-0600 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_s3_object.configurationfolders[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .object_lock_retain_until_date: planned value cty.StringVal("") for a non-computed attribute
      - .content_language: planned value cty.StringVal("") for a non-computed attribute
      - .website_redirect: planned value cty.StringVal("") for a non-computed attribute
      - .metadata: planned value cty.MapValEmpty(cty.String) for a non-computed attribute
      - .object_lock_legal_hold_status: planned value cty.StringVal("") for a non-computed attribute
      - .cache_control: planned value cty.StringVal("") for a non-computed attribute
      - .object_lock_mode: planned value cty.StringVal("") for a non-computed attribute
      - .force_destroy: planned value cty.False for a non-computed attribute
      - .content_encoding: planned value cty.StringVal("") for a non-computed attribute
      - .content_disposition: planned value cty.StringVal("") for a non-computed attribute
2023-06-02T08:37:28.731-0600 [DEBUG] ProviderTransformer: "aws_s3_object.configurationfolders (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]
2023-06-02T08:37:30.055-0600 [DEBUG] ReferenceTransformer: "aws_s3_object.configurationfolders (expand)" references: [local.all_services (expand)]
2023-06-02T08:37:28.717-0600 [DEBUG] ProviderTransformer: "aws_s3_object.configurationfolders[0]" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/hashicorp/aws"]
2023-06-02T08:37:28.731-0600 [DEBUG] ProviderTransformer: "aws_s3_object.configurationfolders (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/hashicorp/aws"]

terraform apply:

2023-06-02T08:43:08.823-0600 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_s3_object.configurationfolders[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .content_encoding: planned value cty.StringVal("") for a non-computed attribute
      - .object_lock_retain_until_date: planned value cty.StringVal("") for a non-computed attribute
      - .content_disposition: planned value cty.StringVal("") for a non-computed attribute
      - .force_destroy: planned value cty.False for a non-computed attribute
      - .object_lock_legal_hold_status: planned value cty.StringVal("") for a non-computed attribute
      - .object_lock_mode: planned value cty.StringVal("") for a non-computed attribute
      - .cache_control: planned value cty.StringVal("") for a non-computed attribute
      - .metadata: planned value cty.MapValEmpty(cty.String) for a non-computed attribute
      - .content_language: planned value cty.StringVal("") for a non-computed attribute
      - .website_redirect: planned value cty.StringVal("") for a non-computed attribute
2023-06-02T08:43:08.823-0600 [INFO]  Starting apply for aws_s3_object.configurationfolders[0]
2023-06-02T08:43:08.825-0600 [DEBUG] aws_s3_object.configurationfolders[0]: applying the planned Update change
2023-06-02T08:43:09.166-0600 [ERROR] vertex "aws_s3_object.configurationfolders[0]" error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
    {
      "mode": "managed",
      "type": "aws_s3_object",
      "name": "configurationfolders",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 0,
          "attributes": {
            "acl": "",
            "bucket": "xxx",
            "bucket_key_enabled": false,
            "cache_control": "",
            "content": "xxx",
            "content_base64": null,
            "content_disposition": "",
            "content_encoding": "",
            "content_language": "",
            "content_type": "binary/octet-stream",
            "etag": "xxx"
            "force_destroy": false,
            "id": "xxx",
            "key": "xxx",
            "kms_key_id": xxx,
            "metadata": {},
            "object_lock_legal_hold_status": "",
            "object_lock_mode": "",
            "object_lock_retain_until_date": "",
            "server_side_encryption": "xxx",
            "source": null,
            "source_hash": null,
            "storage_class": "STANDARD",
            "tags": {},
            "tags_all": {},
                      "sensitive_attributes": [],
          "private": "bnVsbA==",
          "dependencies": [
            "aws_s3_bucket.xxxx"
          ]
        },
╷
│ Error: putting S3 object ACL: MissingSecurityHeader: Your request was missing a required header
│       status code: 400, request id: xxxx, host id: xxxxxxxx
│
│   with aws_s3_object.configurationfolders[0],
│   on xxx.tf line, in resource "aws_s3_object" "configurationfolders":
│ xxx: resource "aws_s3_object" "configurationfolders" {
│

naomichi-y · 2023-07-22T06:38:44Z

I have the same problem.
By adding acl = "private" to aws_s3_object, I was able to temporarily avoid it.
Just to be sure, I checked the ACL status with get-object-acl, but it doesn't seem to have changed.

ewbankkit · 2023-08-17T16:11:37Z

Relates #27197.

github-actions · 2023-08-24T21:11:31Z

This functionality has been released in v5.14.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

robinpecha · 2023-08-30T10:36:08Z

Running terraform apply twice fixed this issue for me.

tmetn · 2023-09-15T16:00:03Z

Any long term solution for this?

grimm26 · 2023-09-16T03:44:15Z

Any long term solution for this?

yes. #31633 (comment)

github-actions · 2023-10-17T02:02:36Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

quentin9696 added bug Addresses a defect in current functionality. needs-triage Waiting for first response or review from a maintainer. labels May 29, 2023

github-actions bot added the service/s3 Issues and PRs that pertain to the s3 service. label May 29, 2023

kpitzen mentioned this issue Jun 1, 2023

[Epic] Pulumi AWS 6.0 pulumi/pulumi-aws#2539

Closed

34 tasks

justinretzolk added waiting-response Maintainers are waiting on response from community or contributor. and removed needs-triage Waiting for first response or review from a maintainer. labels Jun 1, 2023

github-actions bot removed the waiting-response Maintainers are waiting on response from community or contributor. label Jun 2, 2023

thoeng-nuuday mentioned this issue Jun 6, 2023

[Bug]: updating s3_object with aws provider 4.67.0 fails with putting S3 object ACL: AccessControlListNotSupported: The bucket does not allow ACLs #31764

Open

GuptaNavdeep1983 mentioned this issue Jul 11, 2023

Lambda Function (scale_up) issue, upgrading to v3.5.0 from 3.4.0 philips-labs/terraform-aws-github-runner#3358

Closed

justinretzolk mentioned this issue Jul 31, 2023

[Bug]: upgrading from 4.67.0 to 5.10.0 cause error: MissingSecurityHeader: Your request was missing a required header #32772

Closed

justinretzolk added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Aug 1, 2023

ewbankkit self-assigned this Aug 17, 2023

ewbankkit mentioned this issue Aug 22, 2023

AWS S3 Object: Support seamless upgrade from provider v4.x #33138

Merged

ewbankkit closed this as completed in #33138 Aug 22, 2023

github-actions bot added this to the v5.14.0 milestone Aug 22, 2023

github-actions bot removed the bug Addresses a defect in current functionality. label Aug 24, 2023

github-actions bot locked as resolved and limited conversation to collaborators Oct 17, 2023

justinretzolk added the bug Addresses a defect in current functionality. label Feb 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Migrate aws_s3_object from provider 4 to 5 is not working at first apply #31633

[Bug]: Migrate aws_s3_object from provider 4 to 5 is not working at first apply #31633

quentin9696 commented May 29, 2023 •

edited by justinretzolk

Loading

github-actions bot commented May 29, 2023

justinretzolk commented Jun 1, 2023

pguinard-public-com commented Jun 2, 2023 •

edited

Loading

naomichi-y commented Jul 22, 2023 •

edited

Loading

ewbankkit commented Aug 17, 2023

github-actions bot commented Aug 24, 2023

robinpecha commented Aug 30, 2023

tmetn commented Sep 15, 2023

grimm26 commented Sep 16, 2023

github-actions bot commented Oct 17, 2023

[Bug]: Migrate aws_s3_object from provider 4 to 5 is not working at first apply #31633

[Bug]: Migrate aws_s3_object from provider 4 to 5 is not working at first apply #31633

Comments

quentin9696 commented May 29, 2023 • edited by justinretzolk Loading

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

github-actions bot commented May 29, 2023

Community Note

justinretzolk commented Jun 1, 2023

pguinard-public-com commented Jun 2, 2023 • edited Loading

naomichi-y commented Jul 22, 2023 • edited Loading

ewbankkit commented Aug 17, 2023

github-actions bot commented Aug 24, 2023

robinpecha commented Aug 30, 2023

tmetn commented Sep 15, 2023

grimm26 commented Sep 16, 2023

github-actions bot commented Oct 17, 2023

quentin9696 commented May 29, 2023 •

edited by justinretzolk

Loading

pguinard-public-com commented Jun 2, 2023 •

edited

Loading

naomichi-y commented Jul 22, 2023 •

edited

Loading