-
Notifications
You must be signed in to change notification settings - Fork 9.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS security groups not being destroyed #2445
Comments
is the SG attached to anything like a ELB or instance? |
Yeah. It's attached to one instance managed by Terraform. If I manually remove association everything goes well, but terraform does not do that on its own. |
Hi @szczad! Sorry for this limitation. At the moment Terraform doesn't have any mechanism to deal with these "enforced dependencies" in the underlying service, and indeed it affects a number of resources, such as what we see in #646, #151, #2201. The problem is that this requires some extra coordination between operations -- creating multiple coordinated steps as you mentioned -- which Terraform's provider model isn't currently able to represent. In certain cases such dependencies could be represented in principle -- for example, in this case where Terraform is managing both the security group and the other resources that belong to it -- while in other cases Terraform can't "see" the dependency at all, because e.g. the EC2 instances are being created implicitly by an We would like to find a way to address this limitation in the long run, for certain situations at least, but at this time we do not have a suitable design figured out to deal with it, and our current Terraform Core development focus is elsewhere. For now it is, as you noted, required to manually break the dependency somehow before making these changes, which is definitely not ideal. In principle we could improve the behavior here by at least producing a helpful error when this situation arises. Unfortunately the long-running polling behavior you saw here was introduced to work around a different problem: network interfaces tend to live for a few minutes after their associated resources are destroyed, acting as dependencies on the security group that aren't reflected in the API at all. Terraform therefore retries here so that it can wait until VPC has finished deallocating the network interface before proceeding, rather than failing with a hard error in that case. |
Hmm... That sounds complicated a lot. But thanks for detailed explanation. |
I often run into this exact issue, which is quite annoying, especially when I'm trying to Having a dependency violation come up, when I'm trying to get rid of my resources, is not expected behavior. Does this mean that the |
Has 0.12 introduced anything that'd allow for a fix to this? Having to manually alter >80 security groups via the console is not a pleasant experience. |
Also related (SG with RDS, SG with VPCE): #9692 |
This is also manifest by removing a security group AND its association with any instances. In this case, the dependency is broken by the change to the instance, but terraform is trying to delete the security group BEFORE modifying the instance. Just another manifestation of the same issue. |
I experience this problem with the following resources:
I am using: When I attempt to destroy the resources, the security group hangs for several minutes. When investigating in the UI, it appears there is a network interface dependency which prevents it's deletion from the AWS console. If I delete the network interface manually, the terraform deletion succeeds very quickly. If I do not, ultimately the operation fails similarly.
|
Same issue here. When I try to apply a change to a SG that forces replacement:
|
What are the workarounds for this? We're evaluating Terraform right now to potentially switch from using Ansible for provisioning cloud infrastructure, but this seems like a pretty glaring omission. Is there really no way to tell Terraform to remove the SG from places it is used before updating it? I can't imagine heavy users of Terraform are continuing to make manual configuration changes. Thanks for your help |
To add to this thread. I often find upon a second run of the |
I'm getting this as well. Are there any workarounds? |
This issue occurs when renaming a security group in Terraform and also updating the Terraform tries to do the following:
This causes (1) to hang because the AWS API prevents deleting a SG that's still associated with an instance. (2) succeeds, and (3) is never executed. The correct order of actions should be:
I didn't find any working solutions, except during the hanging going to the AWS Management Console Actions > Networking > Change Security Groups, disassociating the old security group, and associating the new security group. |
Hitting the same: |
With create_before_destroy = true in SG it works.
|
Thanks for this! I'm watching a SG try to delete for over 60 minutes now - painstakingly went through discrete steps of creating a new group, assigning the new group, deleting the old group - oh wait, I tested manually adding a Lambda function to the SG and removing it, but now the SG is still attached to those network interfaces and won't delete ಠ_ಠ
|
Terraform v0.12.26
The issue is the the default SG is detroyed BEFORE the EC2 SG is and in fact it's not true - the default SG is still there even if Terraform says "destroyed" (see 2nd line below)
The GC-SG-VPC-test security group is part the the default SG rules !!! |
Thanks @BcTpe4HbIu
worked in my case. |
This worked for me:
to the old security group while renaming it.
|
this is what I am trying to avoid - doing things manually in the console. |
Hi everyone, We know this is a frustrating issue for you all. Unfortunately, the Terraform dependency model doesn't yet support bi-directional dependencies between resources that would allow general modifying other resources as part of deletion or modification. We have an open issue on Terraform core to address this. There is a general workaround for this dependency case, described at hashicorp/terraform#16065 (comment), though it may not be applicable here. One additional workaround that may work in some of your cases with Since this issue requires changes to the core Terraform dependency model, I'm going to close this issue. Once the support is available, we will address this and other issues caused by dependencies across resources. You may be able to find other workarounds or solutions in our forums for the AWS Provider or Terraform. |
So Terraform gives up on that - woow - i can't believe it |
Our scenario/workaround: TF couldn't destroy/replace a security group because it was still attached to an ALB. We had to do 3 separate TF runs:
This avoids horrid manual console/statefile intervention. Hope this helps someone! |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
Hi there,
Terraform Version
Terraform v0.11.0
Affected Resource(s)
Please list the resources as a list, for example:
(probably more)
Terraform Configuration Files
Changed "name" from "all-zabbix" to "all-zabbix-test"
Debug Output
Expected Behavior
Actual Behavior
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
terraform apply
The text was updated successfully, but these errors were encountered: