aws_cloudwatch_metric_stream resource tagging always shows changes

[joncolby]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

terraform version
Terraform v1.0.0
on darwin_amd64

aws provider version

- Installing hashicorp/aws v3.69.0...
- Installed hashicorp/aws v3.69.0 (signed by HashiCorp)

Affected Resource(s)

• aws_cloudwatch_metric_stream

Terraform Configuration Files

We are using the default_tags feature of the aws provider. This may somehow play a part in this bug.

# aws provider using default_tags
provider "aws" {
  region              = "eu-west-1"
  default_tags {
    tags = 
      {
          "application_name" = "obfuscated"
          "environment"      = "obfuscated"
          "product_name"     = "obfuscated"
          "team_name"        = "obfuscated"
      }
  }
}

resource "aws_cloudwatch_metric_stream" "main" {
  name          = "my-metric-stream"
  role_arn      = aws_iam_role.metrics_stream.arn
  firehose_arn  = aws_kinesis_firehose_delivery_stream.this.arn
  output_format = var.metric_stream_output_format
  tags          = var.tags

 # we allow users to select which cloudwatch namespaces to include using a list variable
  dynamic "include_filter" {
    for_each = toset(var.metric_namespaces_default)

    content {
      namespace = include_filter.key
    }
  }

}

Debug Output

Panic Output

Expected Behavior

tags and tags_all show pending changes with multiple runs of plan/apply (with no code changes to tags between runs

)

Actual Behavior

Running terraform plan or terraform apply multiple times, with no code changes to the resources, always shows pending changes.

Terraform will perform the following actions:

  # module.newrelic_integration.aws_cloudwatch_metric_stream.main will be updated in-place
  ~ resource "aws_cloudwatch_metric_stream" "main" {
        id               = "my-metric-stream"
        name             = "my-metric-stream"
      ~ tags             = {
          - "a"      = "bar" -> null
          - "b"        = "foo" -> null
          - "c" = "hello" -> null
        }
      ~ tags_all         = {
          - "a"      = "bar" -> null
          - "b"        = "foo" -> null
          - "c" = "hello" -> null
          + "d"            = "new"
            # (5 unchanged elements hidden)
        }
        # (7 unchanged attributes hidden)

        # (12 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Steps to Reproduce

1. terraform apply

Important Factoids

We are using the default_tags feature of the aws provider. This may somehow play a part in this bug.

Observation:

It seems that tags_all is not saved to state. I tried to set the resources tags = {} , to set tags only with the default_tags on the aws provider config. However this problem still persists: All the tags in tags_all will show being added, every time plan or apply is run.

# aws provider using default_tags
provider "aws" {
  region              = "eu-west-1"
  default_tags {
    tags = 
      {
          "application_name" = "obfuscated"
          "environment"      = "obfuscated"
          "product_name"     = "obfuscated"
          "team_name"        = "obfuscated"
      }
  }
}

  # module.newrelic_integration.aws_cloudwatch_metric_stream.main will be updated in-place
  ~ resource "aws_cloudwatch_metric_stream" "main" {
        id               = "my-metric-stream"
        name             = "my-metric-stream"
        tags             = {}
      ~ tags_all         = {
          + "application_name" = "obfuscated"
          + "environment"      = "obfuscated"
          + "product_name"     = "obfuscated"
          + "team_name"        = "obfuscated"
        }

References

• #0000

[joncolby]

update: workaround , if anyone is running into this, is to use ignore_changes = [tags_all]

[bwhaley]

This is probably happening because tags on metric streams can only be applied when creating, not during updates. As per the AWS docs:

You can use this parameter only when you are creating a new metric stream. If you are using this operation to update an existing metric stream, any tags you specify in this parameter are ignored. To change the tags of an existing metric stream, use TagResource or UntagResource.

I ended up manually running aws cloudwatch tag-resource to tag an existing metric stream and the diff went away.

Might want to mention it in the aws_cloudwatch_metric_stream documentation.

[DePinto]

I don't think this is limited to using the default_tags feature on the provider. We aren't using that feature and are running into a similar issue with this bug.

We just want to simply update our tags, but the tags are never actually updated, even though the apply shows as successful.
So every subsequent run shows the change taking place.

I think this is because since we are supply tags https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/service/cloudwatch/metric_stream.go#L167-L168 is never hit, and the update never occurs.

[github-actions]

This functionality has been released in v4.47.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.