aws_s3_bucket_object not working with more than 10 default tags

[chris922]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When you define more than 10 default tags in your AWS provider it is not possible to use the aws_s3_bucket_object resource, because AWS doesn't allow more than 10 tags for S3 objects.

I guess this problem should be solved on a higher level, e.g. introduce a possibility to exclude certain or all default tags for any resource or exclude on AWS provider level certain resources for default tags.

Workaround: Creating an additional AWS provider with an alias that do not have any default-tags. :(

Terraform CLI and Terraform AWS Provider Version

Terraform v0.15.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v3.50.0

Affected Resource(s)

• aws_s3_bucket_object

Terraform Configuration Files

provider "aws" {
  region = "eu-central-1"
  
  default_tags {
    tags = {
      tag1 = "value1"
      tag2 = "value2"
      tag3 = "value3"
      tag4 = "value4"
      tag5 = "value5"
      tag6 = "value6"
      tag7 = "value7"
      tag8 = "value8"
      tag9 = "value9"
      tag10 = "value10"
      tag11 = "value11"
    }
  }
}

resource "aws_s3_bucket" "bucket" {
  name = "foo"
}

resource "local_file" "foo" {
  content = "Lorem ipsum"

  filename = "test_file.txt"
}

resource "aws_s3_bucket_object" "object" {
  bucket = aws_s3_bucket.bucket.bucket
  
  key = "bar"
  
  source = local_file.foo.filename
  etag = filemd5(local_file.foo.filename)
}

Debug Output

Error: Error putting object in S3 bucket (foo): BadRequest: Object tags cannot be greater than 10

Panic Output

Expected Behavior

It is possible to create the s3_object

Actual Behavior

It is not possible to create the s3_object

Steps to Reproduce

1. terraform init && terraform apply

Important Factoids

References

[ewbankkit]

There is a hard limit of 10 tags per S3 Object: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-tagging.html.

[erikpaasonen]

There is a hard limit of 10 tags per S3 Object: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-tagging.html.

that's correct. should this resource type take the first 10 tags from the provider's default_tags to avoid this error?

[chris922]

@erikpaasonen: If this is the most simple solution it could already be a good start, as it would allow using this resource when more than 10 default tags are configured. Right now we are defining the same provider twice, one with default tags, one without... as we are using this resource in some modules to which we have the pass the additional provider this makes it really cumbersome to manage.

Afaik the order for default_tags is not guaranteed, so it can't be determined which tags will be used when always just the "first 10" will be used. Additionally, the regular tags attribute has to be considered as well (imho they have a higher priority than default tags). So maybe the target solutions should be something different. Maybe just an ignore_default_tags property?

[ollytheninja]

Would it be better to have a way to choose which default tags to choose or ignore?
i.e. the ignore_default_tags property takes a list of default tags to ignore?

I've come across this issue because I was looking for issues related to this Terraform provider validating how many tags are passed to AWS. Even without >10 default tags it is cumbersome to have aws_s3_bucket_object inherit the tags I want it to have (using a merge()).

Because of this I would suggest perhaps a filter_tags option to specify which tags should be included.
This would probably be more work to implement, happy to open a new ticket but interested to hear what others are thinking.

[ewbankkit]

Relates: #19895.

[chris922]

@ollytheninja : I would already be fine with a ignore_default_tags boolean flag that I can just set to true and no default tags will be included at all. Maybe there is a nice way to support both? e.g. allow wildcards like * so that you can define one of:

ignore_default_tags = ["foo", "bar"]

ignore_default_tags = ["foo*"]

ignore_default_tags = ["*"]

As we often group tags via prefixes allowing wildcards would allow to ignore whole groups of prefixes :)

[ollytheninja]

@chris922 I think if the filter approach is used then the parameter needs to be filter_default_tags since ignore_default_tags implies a boolean.
I also think we should have the option to filter all tags, not just default tags.
Many organizations have a collection of tags they extend / apply into the tag parameter, being able to filter those also would be ideal.

I guess there are multiple possible solutions here, one or more of which could be applied:

• Static validation in the terraform provider to error sooner.
• Ability to not include default tags
• Ability to filter default tags
• Ability to filter all tags

I've never contributed to Terraform, I'll see if I can find the time in the next few weeks to dive in and understand what would be required to implement this.
Interested to hear others' thoughts on how to overcome this limitation in AWS!

[jwhitcraft]

has there been any movement on this? We just got bit by this, and I'd like a way to just have this ignore the default_tags coming in and just take the tags passed into it.

[ericrichtert]

I missed this issue, so from #24692, I'd like to mention two possible options:

• don't apply default_tags on aws_s3_object if the number of tags > 10
• don't apply default_tags on aws_s3_object

[nvandermark]

We've also run into this issue where I work. Because of our tagging policy, which lists more than 10 required tags.

The workaround we've used is to supply a secondary Provider that has no default tags, but is otherwise identical to the existing one. For those coming to this thread looking for a solution, my starting point was the Terraform documentation on Provider configurations.

Of course, a better solution would be preferred in the long run.

[dzierzanowski]

This issue is still very much relevant.

[jamespfluger-ava]

It's important to mention this is not mentioned in the Terraform documentation either, so it's upon the user to either read the exact AWS documentation or to (more likely) stumble across the error themselves.

[jamespfluger-ava]

The above MR, #33262, would remove the default tags (good) but we need AWS to increase the S3 tagging limit for TF to have this fixed.

[ollytheninja]

@jamespfluger-ava while I zagerer this is on AWS to fix properly, given S3 object tagging was introduced in 2016, three months after other tags were increased from 10 to 50. I for one won't be holding my breath on AWS fixing it anytime soon.

[github-actions]

This functionality has been released in v5.24.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.