aws_instance block_device declaration causes continual resource re-creation #913
Comments
|
Hey @patricklucas - sorry this is causing you trouble. Over in #859 we determined that terraform should ignore root devices, which is the current behavior in master. It looks like your config is declaring a root device - do you get coherent behavior when you only specify the non-root device in your config and let AWS auto-create the other one? |
|
If I remove the /dev/sda1 block_device, it does indeed start to work. But, I need to specify the root device because I need to override its volume type and size—I want gp2/16GB instead of the default standard/8GB. Is this not possible with Terraform? |
|
The solution we landed on in #859 may have been insufficient. Let me think about this a bit. |
|
I see this behavior even when I don't specify the root device. Here's my config: terraform apply forces a new resource, with this relevant output: |
|
In my while the statefile from the last apply has the following mapping registered: The fact that those numeric ids don't match lets me assume that terraform does think it has to recreate the whole device every time as it is a unknown one. Can someone point me to the commit where the diff introduced the numeric id for lists? |
|
Thanks for the feedback folks. We'll get this all untangled. @threetee is your case one where you are adding a block device to an existing instance, or was that I think one important issue here for updates is that we're currently marking every attribute of terraform/builtin/providers/aws/resource_aws_instance.go Lines 142 to 187 in 680fa3c This is because the Update function is pretty anemic right now - it only supports passing @MerlinDMC Same question for you - is this for a new instance or for updates? FWIW we're tracking ephemeral storage support over in #858, though all of this stuff is very intertwined. So teasing out the different issues we have so far:
|
|
@phinze I created a new instance in a clean environment and then ran transscript is: https://gist.github.com/MerlinDMC/5bfd28ccd98eded278d9 using terraform 0.3.6 installed via homebrew config used: |
|
I hope I'm not way off here, but while trying to debug this (with a very limited knowledge of Go) it looks like the delete_on_termination value at https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/resource_aws_instance.go#L513 is being set to |
|
Hey @jschneiderhan - thanks the debugging assist! I think you're right that there's a problem with |
|
As a workaround, if you specify |
|
@jschneiderhan: That doesn't work for me. Even when explicitly setting it to true, terraform tries to refresh the resources on every run. |
|
@patricklucas noted - perhaps it's dependent on the version I'm using, 0.3.5, which is a bit older. The following leads to re-creation of the instance: But this does not: |
|
I just installed v0.3.6 and see the same behavior with the above template. It must be conditional on something, but I don't know what. I need to move on to something else, but if there is anything more I can try to help please let me know. |
@phinze sorry, upon re-reading my post, I can see how it wasn't clear exactly how I arrived at the problem. The /dev/sdf device was in fact there on creation. I just ran through the whole process again to ensure I could still replicate, and here are the exact steps to duplicate for me:
I'm running terraform v0.3.6, installed via homebrew on OS X Yosemite. |
|
I'm having the same issue : |
|
My current workaround is to create the instance with the root block device defined, then remove it for subsequent runs. If you always want the same root device spec, you can also make it your account default. When you launch an instance with Terraform without defining your root device it will use those settings. |
|
@patricklucas this is not an option however as it defies the IaC purpose. |
|
Hey folks, sorry for all the trouble with this. We're seeing this pattern of sadness in several different places where the backing API provides a collection of objects in which a subset has certain special properties - particularly where a subset of the collection can be auto-populated while the rest is managed. In this case, the collection of Today I'm working on a candidate solution for this that I'd love to get feedback on in case it's not the right direction. Basically I'd like to try splitting out the root device into a separate sub-resource like this: resource "aws_instance" "foo" {
ami = "ami-55a7ea65"
instance_type = "m1.small"
root_block_device {
device_name = "/dev/sda1"
volume_size = 10
}
block_device {
device_name = "/dev/sdb"
volume_size = 11
}
block_device {
device_name = "/dev/sdc"
volume_size = 12
}
// ...
}I believe this will make the behavior much clearer. Let me know what you think! |
|
+1 on a Would it be safe to default the device_name to "/dev/sda1" instead of require that a user specify it? The EC2/EBS documentation makes it pretty clear that this is always the default for linux boxes and would prevent mistakes (like the one I just made) where a user specifies In the meantime, until this ships, I'm going with this approach to making the root volume larger: Please let me know if there is a better way for now (with Terraform 0.3.5 or 0.3.6). Thanks! |
|
@phinze suggested approach is very fitting. +1.
|
|
@phinze: sgtm I concur with @joegoggins that device_name can likely be safely defaulted to /dev/sda1, though I believe that volume_size and volume_type should fall back to your account defaults as is the current behavior. |
AWS provides a single `BlockDeviceMapping` to manage three different kinds of block devices: (a) The root volume (b) Ephemeral storage (c) Additional EBS volumes Each of these types has slightly different semantics [1]. (a) The root volume is defined by the AMI; it can only be customized with `volume_size`, `volume_type`, and `delete_on_termination`. (b) Ephemeral storage is made available based on instance type [2]. It's attached automatically if _no_ block device mappings are specified, and must otherwise be defined with block device mapping entries that contain only DeviceName set to a device like "/dev/sdX" and VirtualName set to "ephemeralN". (c) Additional EBS volumes are controlled by mappings that omit `virtual_name` and can specify `volume_size`, `volume_type`, `delete_on_termination`, `snapshot_id`, and `encryption`. After deciding to ignore root block devices to fix #859, we had users with configurations that were attempting to manage the root block device chime in on #913. Terraform does not have the primitives to be able to properly handle a single collection of resources that is partially managed and partially computed, so our strategy here is to break out logical sub-resources for Terraform and hide the BlockDeviceMapping inside the provider implementation. Now (a) is supported by the `root_block_device` sub-resource, and (b) and (c) are still both merged together under `block_device`, though I have yet to see ephemeral block devices working properly. Looking into possibly separating out `ephemeral_block_device` and `ebs_block_device` sub-resources as well, which seem like the logical next step. We'll wait until the next big release for this, though, since it will break backcompat. [1] http://bit.ly/ec2bdmap [2] http://bit.ly/instancestorebytype Fixes #913 Refs #858
AWS provides a single `BlockDeviceMapping` to manage three different kinds of block devices: (a) The root volume (b) Ephemeral storage (c) Additional EBS volumes Each of these types has slightly different semantics [1]. (a) The root volume is defined by the AMI; it can only be customized with `volume_size`, `volume_type`, and `delete_on_termination`. (b) Ephemeral storage is made available based on instance type [2]. It's attached automatically if _no_ block device mappings are specified, and must otherwise be defined with block device mapping entries that contain only DeviceName set to a device like "/dev/sdX" and VirtualName set to "ephemeralN". (c) Additional EBS volumes are controlled by mappings that omit `virtual_name` and can specify `volume_size`, `volume_type`, `delete_on_termination`, `snapshot_id`, and `encryption`. After deciding to ignore root block devices to fix hashicorp#859, we had users with configurations that were attempting to manage the root block device chime in on hashicorp#913. Terraform does not have the primitives to be able to properly handle a single collection of resources that is partially managed and partially computed, so our strategy here is to break out logical sub-resources for Terraform and hide the BlockDeviceMapping inside the provider implementation. Now (a) is supported by the `root_block_device` sub-resource, and (b) and (c) are still both merged together under `block_device`, though I have yet to see ephemeral block devices working properly. Looking into possibly separating out `ephemeral_block_device` and `ebs_block_device` sub-resources as well, which seem like the logical next step. We'll wait until the next big release for this, though, since it will break backcompat. [1] http://bit.ly/ec2bdmap [2] http://bit.ly/instancestorebytype Fixes hashicorp#913 Refs hashicorp#858
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
I am unable to use latest (c18b01f) Terraform to manage AWS instances because it tries to destroy and re-create them on every invocation. I think it has something to with my block device specifications. I have already launched quite a few instances with Terraform, but I can't make any modifications lest they get destroyed.
Here is the Terraform config and output:
The text was updated successfully, but these errors were encountered: