aws_instance volume_tags conflict with aws_ebs_volume tags for attached volumes in a module

[hashibot]

This issue was originally opened by @kojiromike as hashicorp/terraform#14462. It was migrated here as part of the provider split. The original body of the issue is below.

---

Terraform Version

0.9.5

Affected Resource(s)

Please list the resources as a list, for example:

• aws_instance
• aws_ebs_volume
• aws_volume_attachment

Terraform Configuration Files

1. A file in a non-root module:

resource "aws_instance" "test" {
  ami               = "ami-20328b40"
  availability_zone = "us-west-2a"
  instance_type     = "t2.micro"
  subnet_id         = "subnet-850585e2"
  key_name          = "michaels_id_rsa"

  tags {
    Name = "test-instance"
  }

  volume_tags {
    Name = "test-instance"
  }
}

variable "vol_id" {}

resource "aws_volume_attachment" "test" {
  device_name = "/dev/xvdf"
  instance_id = "${aws_instance.test.id}"
  volume_id   = "${var.vol_id}"
}

2. A root module referencing that file:

provider aws {
  region = "us-west-2"
}

resource "aws_ebs_volume" "test" {
  availability_zone = "us-west-2a"
  size              = "1"
  type              = "gp2"

  tags {
    Name = "test-vol"
  }
}

module "test" {
  source = "../../modules/bug"
  vol_id = "${aws_ebs_volume.test.id}"
}

Expected Behavior

The tag for aws_ebs_volume.test should consistently be "Name=>vol" according to the tags specified in that resource.

Actual Behavior

The "Name" tag switches back and forth between the value supplied by the aws_ebs_volume and the aws_instance.volume_tags.

Steps to Reproduce

After each subsequent terraform plan/apply, then I do aws --region us-west-2 ec2 describe-volumes --volume-ids "$volid" | jq -c '.Volumes[].Tags' using the volume id grabbed from the first run:

Run 1: [{"Value":"test-vol","Key":"Name"}]
Run 2: [{"Value":"test-instance","Key":"Name"}]

Important Factoids

This is so similar to #729 I almost requested that be reopened, but when I tried to reproduce it in a very simple case without a non-root module, I couldn't. But when I put some components in a module, the problem was evident.

[morissm]

Am I the only one who feels that the whole idea behind volume_tags is conceptually deficient and will forever be bug-prone? It seems to me that root_block_device and ebs_block_device should instead have tags arguments. That would provide a clean, clear and more flexible way to achieve the same goal.

Is it too late to re-design this feature?

[syquus]

@morissm no, you aren't. This smells like something not very well designed. What I am not sure yet is where the wrong is, if it's something regarding terraform implementation itself, or it's something inherent to AWS resources internal design...

[morissm]

@syquus Oh I'm not talking about the implementation but about the very design philosophy behind volume_tags.

This argument conceptually impacts the configuration of multiple resources: the root volume, the ebs_block_devices defined inline in the aws_instance and the aws_ebs_volumes attached to the instance which may or may not define their own sets of tags.

This goes against the general design of Terraform where the configuration of each resource is solely determined by its arguments. Dependencies are expressed solely through interpolations made explicit in the config with no second order effects.

volume_tags flies in the face of this philosophy, even inside aws_instance resources which is why I believe it will always be bug prone in its current design.

I don't have time to fix this myself but if it were me, I would not fix volume_tags, add a tags argument to ebs_block_device and root_block_device now, and sunset volume_tags in the next major release.

[mochi99999]

I agree to adding tags to root_block_device. However, I've noticed that any changes to root_block_device will cause the instance resource to be rebuilt. I'm not sure if this would be an issue if we're adding tags to an existing root_block_device

edit: I'm not sure why my comment unassigned @stack72. my apologies.

[s-nakka]

I still see the issue

[cmhamill]

Since this seems to be sitting around without a clear path forward, I'll share my workaround which I'm reasonably content with:

resource "aws_instance" "foo" {
  ...
  lifecycle {
    ignore_changes = [
      "ami",
      "key_name",

      # This prevents clobbering the tags of attached EBS volumes. See
      # [this bug][1] in the AWS provider upstream.
      #
      # [1]: https://github.com/terraform-providers/terraform-provider-aws/issues/770
      "volume_tags",
    ]
  }

  tags = "${merge(local.tags, map(
    "Name", "${local.name_prefix}-foo",
    "Role", "foo",
  ))}"

  volume_tags = "${merge(local.tags, map(
    "Name", "${local.name_prefix}-foo-rootfs",
    "Role", "foo-rootfs",
  ))}"

In addition, I avoid putting any EBS configuration other than the root block device into the instance resource itself, and tag aws_ebs_volume resources separately, however I feel inclined. This seems to break the logic that applies the volume_tags to all attached EBS volumes, and basically makes volume_tags only apply to the root device.

This isn't so much a workaround as a way to tag root volumes and otherwise ignore the feature. My conclusions:

1. It's conceivable the entire feature should be dropped.
2. Perhaps there should be a tags key under the root_block_device and block_device stanzas in the aws_instance resource.
3. Perhaps volume_tags should be renamed default_volume_tags, and gain an inheritance relationship with tags specified in aws_ebs_volume resources. If option 2 above is also pursued, then it could be overridden by those, also.

Just my 2¢.

[tdmalone]

Related tickets: #729 & #884 (possibly all duplicates of the same issue?)

[rastakajakwanna]

Still an issue for

Terraform v0.11.10
+ provider.aws v1.41.0
+ provider.template v1.0.0

[dekimsey]

I was hoping terraform 0.12's support for null resources would help address this, but unfortunately it does not. Even when set to null it flip-flops :(

I for one vote for the removal of the feature or at least not affecting volumes attached by other means.

[slessardjr]

This is a nightmare for trying to edit root volume options like Name, or even adding a tag for a lifecycle policy within AWS. I attempted to edit just the root device (all attached EBSs are really just Kubernetes Persistent Volumes) and I can't. This feel very broken, and I don't get why after 2+ years of reporting this that nothing has been done.

[sergikpas]

Guys, any update on it? It's creates a real headache.

[YakDriver]

We have merged a fix to the volume_tags issue in #15474. We have added tests to cover the issues observed. Please note that using volume_tags in aws_instance is not compatible with using tags in aws_ebs_volume. You need to use one or the other. Prior to this fix, even following this rule, you would encounter errors. Along with the fix, we've added tags to the root_block_device and ebs_block_device configuration blocks in aws_instance.

Now that the fix is in place, if you find any problems with volume_tags, let us know by opening a new issue.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!