AWS provider 4.10.0 - Order is lost for Bucket policy

[Mornor]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.8
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.10.0

Affected Resource(s)

• aws_s3_bucket
• aws_s3_bucket_policy

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Account IDs
variable "account_ids" {
  description = "Map of AWS account IDs."
  type            = map(string)
}

# Account IDs
account_ids = {
  "account_1"  = "xxxx",
  "account_2" = "yyyy"
}


# S3 Bucket Policy
data "aws_iam_policy_document" "lambda-bucket" {
  statement {
    effect = "Allow"

    actions = [
      "s3:*",
    ]

    principals {
      type = "AWS"

      identifiers = [
        "arn:aws:iam::${var.account_ids.account_1}:role/Teamcity-fullaccess",
        "arn:aws:iam::${var.account_ids.account_2}:role/teamcity-poweruser"
      ]
    }

    resources = [
      aws_s3_bucket.lambda-bucket.arn,
      "${aws_s3_bucket.lambda-bucket.arn}/*",
    ]
  }
}

# Bucket
resource "aws_s3_bucket" "lambda-bucket" {
  bucket = "lambdas-${terraform.workspace}"
  tags = merge(var.default_tags, local.env_tag, {
    Name = "lambdas-${terraform.workspace}"
  })
}

resource "aws_s3_bucket_versioning" "lambda-bucket" {
  bucket = aws_s3_bucket.lambda-bucket.bucket
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "lambda-bucket" {
  bucket = aws_s3_bucket.lambda-bucket.bucket

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_acl" "lambda-bucket" {
  bucket = aws_s3_bucket.lambda-bucket.bucket
  acl    = "private"
}

resource "aws_s3_bucket_policy" "lambda-bucket" {
  bucket = aws_s3_bucket.lambda-bucket.id
  policy = data.aws_iam_policy_document.lambda-bucket.json
}

Expected Behavior

aws_s3_bucket.lambda-bucket: Refreshing state... [lambdas-dev1]
aws_s3_bucket_server_side_encryption_configuration.lambda-bucket: Refreshing state... [id=lambdas-dev1]
aws_s3_bucket_acl.lambda-bucket: Refreshing state... [id=lambdas-dev1,private]
aws_s3_bucket_versioning.lambda-bucket: Refreshing state... [id=lambdas-dev1]
aws_s3_bucket_policy.lambda-bucket: Refreshing state... [id=lambdas-dev1]

No changes. Your infrastructure matches the configuration.

Actual Behavior

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

 # aws_s3_bucket.lambda-bucket has changed
  ~ resource "aws_s3_bucket" "lambda-bucket" {
        id                          = "lambdas-dev1"
      ~ policy                      = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                      ~ Principal = {
                          ~ AWS = [
                              - "arn:aws:iam::xxxx:role/Teamcity-fullaccess",
                                "arn:aws:iam::yyyy:role/teamcity-poweruser",
                              + "arn:aws:iam::xxxx:role/Teamcity-fullaccess",
                            ]
                        }
                        # (3 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }
        )
        tags                        = {
            "Environment"  = "dev1"
            "Name"         = "lambdas-dev1"
            "Terraform"    = "True"
        }
        # (11 unchanged attributes hidden)



        # (3 unchanged blocks hidden)
    }

Steps to Reproduce

1. terraform apply

References

• Order is lost for data aws_iam_policy_document when applied to S3 buckets, iam roles, kms keys, etc #11801
• IAM Policy Principals: Objects have changed outside of Terraform #20665
• kms/key: Preserve order in JSON policy lists #21969

The order of the policies applied to the Bucket keep changing, while no changes have been performed on the resource itself.
This has no impact on the deployed resources (no terraform changes to be performed), but this creates an unnecessary long output.

[jurgen-weber-deltatre]

I am getting principals changing often with this piece of code; https://github.com/cloudposse/terraform-aws-elasticsearch/blob/9f3dd591f024d2192544d719737effb7cda19a66/main.tf#L84-L104 and others.

data aws_iam_policy_document node-termination-handler-sqs-policy {
  statement {
    principals {
      type        = "Service"
      identifiers = toset([
        "events.amazonaws.com",
        "sqs.amazonaws.com"
      ])
    }
    actions = [
      "sqs:SendMessage"
    ]
    resources = [
      aws_sqs_queue.node-termination-handler.arn
    ]
  }
}

  # module.addons["01"].aws_sqs_queue.node-termination-handler[0] has changed
!   resource "aws_sqs_queue" "node-termination-handler" {
        id                                = "https://sqs.ap--2.amazonaws.com/xxx/node-termination-handler"
        name                              = "node-termination-handler"
!       policy                            = jsonencode(
!           {
!               Statement = [
!                   {
!                       Principal = {
!                           Service = [
-                               "events.amazonaws.com",
                                "sqs.amazonaws.com",
+                               "events.amazonaws.com",
                            ]
                        }
                        # (4 unchanged elements hidden)
                    },
                ]
                # (1 unchanged element hidden)
            }

but it is inconsistent, maybe happens 40% of the time.

Versions:

Terraform v1.2.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.22.0

[YakDriver]

Also related #18878, #22030

Thanks for reporting this! We're sorry that the last fix (#21997) didn't fully resolve this. We are again making changes that should 🤞 fix this (#28855). However, since we cannot reproduce all scenarios, it's possible it will still not be fully solved. After v4.50.0 is available, if you're still seeing issues, please let us know! Unfortunately, this has taken various stabs over many months, and is surprisingly problematic to reproduce and fix.

[github-actions]

This functionality has been released in v4.50.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.