Default tags for all resources

[flowirtz]

Description

I'd like to set default tags for all resources created within a Terraform stack.

When creating service stacks on AWS with Terraform, I like to tag all resources of a project. Right now, this involves a lot of manual work.
I would like to set default tags with the provider, that then automatically get applied to all AWS resources that support it.

There has been an issue over on the main terraform repo (since 2015) which had plenty of 👍 and got closed in favor of creating issues with the respective providers. I couldn't find such an issue though, thus creating this one.

New or Affected Resource(s)

• aws_*

Potential Terraform Configuration

provider "aws" {
  # ...

  default_tags = {
    Project = "Project_1"
    Environment = "Production"
  }
}

References

• Original issue: Default tags for all resources terraform#2283

---

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[gwkunze]

In general (with my limited knowledge on the provider specifics) this should be possible. The one tricky part would be any resources that have additions to the basic tag properties. The one I know that does this is the auto scaling group resources which in addition to a key and value also has a required property which determines whether the tags propagate to instances the ASG creates. So that should probably be taken into account when (if?) implementing this.

[flowirtz]

@gwkunze sorry for the late reply, am currently on a long vacation, coming back next week and want to have a proper look at this then.

I've never handled the terraform core nor written any plugins, but want to give this a try anyways as I'd love to have this feature.

Is there anything else that you can think of that one would need to keep in mind when implementing this? I'm very confident that my TF knowledge is way more limited than yours haha :-)

Thanks already for this suggestion!

[flowirtz]

Alright, so I investigated this a little further and am pretty sure that I won't have the resources to tackle this. I have no prior experience with go, never contributed to this repo before and have not a whole lot of time.
I'm happy to help somebody else with minor things but that's all I can do, I think.

[jonscheiding]

I'm really interested in this and might be able to put some time into it in the next couple of weeks. I was thinking a little bit on @gwkunze's comment about resources like ASG, and wanted to throw out a couple of potential syntax options.

1. Handle the extra properties in the provider configuration:

    provider "aws" {
      # ...
   
      default_tags = [
        {
          key   = "Environment"
          value = "Development"
    
          aws_autoscaling_group {
            propagate_at_launch = true
          }
        },
        {
          key   = "Project"
          value = "Test Project"
        }
      ]
    }
   

2. Handle the extra properties in the resource configuration:

    resource "aws_autoscaling_group" "asg" {
      # ...
    
      default_tags_propagate_at_launch = true
    }
   

3. Don't handle them at all; ASG's don't get default tags applied, you just have to specify them explicitly as normal tags.

There don't appear to be any other resources where this happens, so it's hard to generalize a decent syntax. But I lean towards #2 as the cleanest way to deal with the fact that propagate_at_launch is required - just make this property required too. (Or maybe required only when there are default tags, if that's possible.)

Thoughts?

[kadrach]

There don't appear to be any other resources where this happens, so it's hard to generalize a decent syntax.

In terms of tag propagation, there are other "AWS" resources where this is supported (though not sure how this is done in terraform):

• CloudFormation propagates all tags applied to a stack to resources it contains on creation (this behaviour cannot be changed as far as I am aware).
• ECS can propagate tags from task definitions to the service (the parameter is even called propagateTags.

[gwkunze]

@jonscheiding my first instinct is to prefer the first option to keep all that configuration together, but on the other hand, the auto-scaling group is just one exception and even there it has become less necessary as EC2 Launch Templates gives you more than enough to replace the need for the "propagateAtLaunch"-feature and using the regular tag syntax, ie:

tags = {
  Foo = "BAR"
}

if much nicer to read/write than

tags = [
  {
    key = "Foo"
    value = "BAR"
  }
]

Even if you give extra properties a reasonable default.

So on second though I think I prefer the second or third option. I can also suggest a potential slight alternative to the second option: specifying a list of tag-keys you want propagated, something like:

resource "aws_autoscaling_group" "asg" {
  propagate_tags = ["Environment", "Project"]
}

[jonscheiding]

@gwkunze If I'm understanding you correctly, you're proposing we go from

resource "aws_autoscaling_group" "asg" {
  tag { 
    key   = "some key"
    value = "some value"
    propagate_at_launch = true
  }
}

to something like

resource "aws_autoscaling_group" "asg" {
  tags = {
    "some key" = "some value"
  }

  propagate_tags_at_launch = ["some key", "Environment"] # You can include default tags here
}

I think I like that syntax better too, but I would shy away from undertaking a breaking schema change.

[jonscheiding]

I've been trying to wrap my head around the Terraform and AWS provider code to understand how this would be implemented. It seems like I would have to touch every resource [that supports tags] and probably add a CustomizeDiff handler that adds the provider-level tags. (For example, here.)

AFAICT there is not any kind of like "provider-level CustomizeDiff" that could be used to do this more DRY-ly.

Can anybody who understands this code better than me confirm that?

[gwkunze]

@jonscheiding I meant going to that syntax only for the "default tag" functionality on the provider itself, not change the existing structure for auto scaling groups. So the change would be something like add a default_tags field on the provider and add propagate_tags_at_launch or maybe propagate_default_tags_at_launch (for more clarity) to asg resources.

[bflad]

Hi, everyone! 👋 Thank you for your interest in this functionality. This would be awesome to have within the Terraform AWS Provider.

I just wanted to drop a quick note in here that the maintainers are on board with thinking about this design and trying to develop this in a way that can be plugged into the existing provider codebase (e.g. per AWS service or resource) with community help if at all feasible. We would likely want to shy away from any large "all or nothing" changes because it will require large amounts of integration testing and hold up other development work within the provider codebase.

Something to also consider here is the related, but opposite problem outlined in hashicorp/terraform#20866 where we could also have the ability to ignore a list of tags, per Terraform AWS Provider.

From an operator perspective, the combined design would hopefully be fairly simple:

# Design sketch - not currently implemented and design may change during development
provider "aws" {
  # ... potentially other configuration ...

  default_tags = {
    Tag1 = "Value1"
    Tag2 = "Value2"
    # ...
  }

  ignore_tags = ["Tag3"]
}

Then any resource could retrieve that information from the meta struct that is available from the provider to appropriately handle adding/removing tags from differences/state values.

However as alluded to above, the underlying implementation work may potentially require some new functionality not currently available (or easily usable) in the Terraform Provider SDK and Terraform AWS Provider codebase. We're hoping to meet about this in the next week or two and can hopefully present more ideas or next steps on the manner. Ideally we would like to have clear documentation on how this can be implemented, documented, and tested so the work can be distributed for each resource when the design work is done.

The aws_autoscaling_group resource will need to be handled specially and will likely require a separate design proposal as it may require potential behavior changes in the resource to align it with the rest of the Terraform AWS Provider resources with regards to resource tagging. 👍

[gwkunze]

While I'm not 100% certain about this, the propagate_tags_at_launch as a list to select which tags to propagate concept proposed above could also be applied to the existing aws_autoscaling_group as a change in interface. At the very least that would normalize tags across all resources making it less of an exception. It could even potentially be made as a backwards-compatible change by having the provider check the type of tags, if it's a list it's in the "old" list-of-objects format and it it's a map(string) it's the same as any other tagged ec2 resource.

I'm somewhat ambivalent about this change as it moves the terraform syntax further away from the raw API which could make it slightly more complex to learn. But on the other hand, consistency is important and it seems with the introduction of Launch Templates Amazon themselves are making "propagate_at_launch" superfluous.

[eesu]

Hi, would it possible to use wildcards with ignore_tags?

Apparently using ignore_changes lifecycle for tags actually works as prefix wilrdcard in Terraform 0.11.x. Would be nice to have similar functionality for 0.12.x

[ghost]

This has been released in version 3.38.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.