Cannot destroy due to cycles when using create_before_destroy

[joekhoobyar]

This should not produce cycles - and it makes the workaround for #2421 unusable for the terraform destroy case.

In other words, we can create infrastructure, but we cannot destroy it!

Here are the files involved, and example output:

https://gist.github.com/joekhoobyar/5421b3ebbf9c313404ec

[joekhoobyar]

#2438 also makes a reference to this issue, but the code in this ticket is a much smaller example yielding the same problem. All you need to do to duplicate this issue is run: terraform destroy

[joekhoobyar]

FWIW - The only workaround I have found is literally to:

• sed -e 's/create_before_destroy = true/create_before_destroy = false/g' -i .bk infrastructure.tf
• terraform destroy
• mv infrastructure.tf.bk infrastructure.tf

Clearly, the create_before_destroy is affecting destroy logic. Since we aren't creating anything, only destroying, there should be no impact at all. Can this be fixed?

[catsby]

Hey @joekhoobyar – sorry for the silence here. Unfortunately, at this time you cannot use the create_before_destroy = true key in the lifecycle block on a resource if it depends on other resources that are not also create_before_destroy = true.

For example, in configuration you shared resource aws_launch_configuration.mobilecloud is create_before_destroy, and depends on aws_iam_instance_profile.mobilecloud, aws_security_group.mobilecloud, and template_file.mobilecloud-bootstrap. Of those three, the template_file resource is the only one that is also create_before_destroy = true.

In order for your example configuration to successfully destroy with no cycles, the same create_before_destroy lifecycle block needs to be added to each resource up the dependency graph.:

• aws_iam_instance_profile.mobilecloud
• aws_iam_role.mobilecloud
• aws_security_group.mobilecloud

This is a known limitation that isn't currently documented, so I opened #2944 in attempt to document it better.

Sorry again for the trouble and the silence, I hope this clears things up!

[catsby]

#2944 documenting this issue was merged, so I'm going to close this issue now. Please let me know if you have further questions.

Thanks!

[gposton]

I just found this after reporting a duplicate issue myself (#3294). This introduces a significant limitation for us:

When the lifecycle rule is not on the ASG, I can change the AMI in the launch config w/o the ASG being destroyed (thus cycling my instances).

When the lifecycle rule is added to the ASG, both the ASG and the launch config is destroyed and re-created. This cycles my instances which happens too quickly for all of our services to initialize and pass health checks.

[roboll]

IMHO this issue deserves a 'real' solution; however, I've had success using the taint command on launch configurations and their autoscaling groups prior to destroy. This triggers terraform to ignore dependency cycles, and eliminates the need to cascade create_before_destroy across the entire manifest for everything the launch configuration depends on. Cascading create_before_destroy for the sake of appeasing the dependency graph isn't really a viable solution, in my experience the performance impact is severe. In this configuration I have create_before_destroy specified on the launch config only, not on the autoscaling group.

terraform apply
terraform taint -module mymodule aws_launch_configuration.mylc
terraform taint -module mymodule aws_autoscaling_group.myasg
terraform destroy

[gposton]

Thanks @roboll for the workaround... and I completely agree w/ your assessment....

Cascading create_before_destroy for the sake of appeasing the dependency graph isn't really a viable solution

Further, cascading 'create_before_destroy' still does not solve the issue. When applying the lifecycle rule to all resources, the ASG is deleted (in addition to the launch configuration) on updates. This results in the instances being prematurely terminated which is not viable.

I'd like to share some background in the hopes that it might help. We've been in an on-going debate about which tool to use for infrastructure automation. I'm a huge advocate of terraform and the Hashicorp suite, but as with most things like this, each person that get's added to the conversation has a different opinion. There's no 'silver bullet' solution, but this issue in particular has hurt my ability to build confidence in terraform as 'the right choice'. I hope this is not taken the wrong way, but what makes it most difficult is that despite many tickets related to this issue, there seems to be no 'traction' in it being addressed.

[rojaro]

This issue should not have been closed. The problem described in this issue is not addressed in any of the other 28 open issues related to create_before_destroy and still present.

Destruction of a deployed stack should not require us to taint all the modules or replacing all occurences of "create_before_destroy = true" with "create_before_destroy = false"

[ljsommer]

This issue is a massive inconvenience that continues to haunt our infrastructure. It usually results in us forcefully and very manually disassembling everything and then starting from scratch.

[joshuaspence]

Is there an issue that I can track for a real solution here? I'll admit that I haven't looked into this issue in any great detail, but it seems like this could be fixed by having Terraform automatically apply create_before_destroy to dependent resources as well? For example, if I specify an aws_launch_configuration (that has create_before_destroy enabled) referencing an aws_security_group, it seems that the situation would be improved if Terraform propagated the create_before_destroy lifecycle attribute to the aws_security_group resource.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.