S3 Bucket replication configuration results in MalformedXML issue in 1.42

[heyitskev1n]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform: 0.11.7
AWS Provider: 1.42.0

Affected Resource(s)

• aws_s3_bucket

Terraform Configuration Files

resource "aws_s3_bucket" "web_primary" {
    bucket = "${var.bucket_name}"
    force_destroy = "true"

  versioning {
	enabled = true
  }

  	replication_configuration {
		role = "${var.replicate_bucket_role_arn}"

		rules {
			prefix = ""
			status = "${var.replicate_bucket_status}"
			destination {
				bucket        = "${aws_s3_bucket.web_backup.arn}"
				storage_class = "STANDARD"
			}
		}
	}
}

Debug Output

Panic Output

Expected Behavior

The apply succeeds.

Actual Behavior

The terraform apply command errors with the following error:

* aws_s3_bucket.web_primary: Error putting S3 replication configuration: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema.

Steps to Reproduce

1. Create the resource initially on the 1.41 aws provider.
2. Attempt to reapply the configuration using the 1.42 aws provider via terraform apply.

Important Factoids

Pinning our AWS provider version to 1.41 allows the terraform apply to succeedagain. Looking in CloudTrail there's a difference in the request parameters of the PutBucketReplication sent to AWS between using the two different provider versions, specifically around the ID and Prefix properties.

1.41:

...
"requestParameters": {
        "replication": [
            ""
        ],
        "bucketName": "<our src bucket name>",
        "ReplicationConfiguration": {
            "Role": "<our role name>",
            "xmlns": "http://s3.amazonaws.com/doc/2006-03-01/",
            "Rule": {
                "Status": "Disabled",
                "Destination": {
                    "Bucket": "<our dest bucket arn>",
                    "StorageClass": "STANDARD"
                },
                "ID": "",
                "Prefix": ""
            }
        }
    }, ...

1.42:

...
"requestParameters": {
    "replication": [
        ""
    ],
    "bucketName": "<our src bucket name>",
    "ReplicationConfiguration": {
        "Role": "<our role name>",
        "xmlns": "http://s3.amazonaws.com/doc/2006-03-01/",
        "Rule": {
            "Status": "Disabled",
            "Destination": {
                "Bucket": "<our dest bucket arn>",
                "StorageClass": "STANDARD"
            }
        }
    }
},...

References

The ReplicationConfiguration docs seem to indicate that "ID" is one of the minimum required fields for a rule but even providing that property on the TF config results in the same issue.
https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-add-config.html

[ewbankkit]

I think it's the missing Prefix value that is causing the problem.
I'll investigate...

[ghost]

With any value in the prefix terraform apply works fine here, without a prefix value it failes as described by @heyitskev1n .

[ewbankkit]

Related:

• r/aws_s3_bucket: Support S3 Cross-Region Replication filtering based on S3 object tags #6095

[ewbankkit]

Unfortunately none of the existing acceptance tests were using an empty prefix 😞.
I can reproduce this with a new test case:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_ReplicationWithoutPrefix'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_ReplicationWithoutPrefix -timeout 120m
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutPrefix
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutPrefix
--- FAIL: TestAccAWSS3Bucket_ReplicationWithoutPrefix (32.97s)
    testing.go:538: Step 0 error: Error applying: 1 error occurred:
        	* aws_s3_bucket.bucket: 1 error occurred:
        	* aws_s3_bucket.bucket: Error putting S3 replication configuration: MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
        	status code: 400, request id: B01E35921C2C94CB, host id: sb+GryOe0m3ANw2IcNXao57E18ZbbGe1qUzi2nkNmHCrN8xFGqLfkzMlOILmK/Eso9GwdHMHmck=
        
        
        
        
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	32.982s
GNUmakefile:20: recipe for target 'testacc' failed
make: *** [testacc] Error 1

[grimm26]

even when I leave out prefix in my config, terraform seems to put in an empty one :(. I think we need to be compliant with CRR v2. Checking out the example replication configs at https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-add-config.html shows that to replicate the whole bucket, just leave out the filter section.

[sqNutrien]

Note: having this problem with Terraform: 0.11.8. Applying a TF config that used to work now fails with the above-referenced error. My replication_configuration block looks like:

replication_configuration {
    role = "${aws_iam_role.jupyterhub-notebooks-replication.arn}"

    rules {
      id = "${var.atlas["org"]}-${var.atlas["env"]}-jupyterhub-notebooks-replication"
      prefix = ""
      status = "Enabled"

      source_selection_criteria {
        sse_kms_encrypted_objects {
          enabled = true
        }
      }

      destination {
        bucket = "${aws_s3_bucket.emr-jupyterhub-nbs-replication-destination.arn}"
        replica_kms_key_id = "${aws_kms_key.replication-key.arn}"
      }
    }
  }

[bflad]

The fix for this has been merged and will release with version 1.43.0 of the AWS provider, likely later today or tomorrow. 👍

[bflad]

This has been released in version 1.43.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!