Terraform is unable to destroy AWS VPC on of EMR cluster

[Ohtar10]

Hi!

I'm using terraform with aws, working with EMR clusters, I'm able to create the cluster completely using the docs instructions but when I try to delete it, terraform always fails in destroy de VPC and I'm forced to go to the console and delete it manually.

Terraform Version

Terraform v0.11.13

Expected Behavior:

Terraform should have destroyed everything including the VPC

Actual Behavior:

The VPC is not deleted even after ~10 min retries, the output of Terraform is:

aws_vpc.spark-emr-main: Still destroying... (ID: vpc-0291df56137e9c5b1, 4m50s elapsed)
2019/04/11 15:05:03 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_vpc.spark-emr-main: DependencyViolation: The vpc 'vpc-0291df56137e9c5b1' has dependencies and cannot be deleted.
	status code: 400, request id: 41465664-4873-4c92-b8b2-2e0bda3a5f19
2019/04/11 15:05:03 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_vpc.spark-emr-main: DependencyViolation: The vpc 'vpc-0291df56137e9c5b1' has dependencies and cannot be deleted.
	status code: 400, request id: 41465664-4873-4c92-b8b2-2e0bda3a5f19
2019/04/11 15:05:03 [ERROR] root: eval: *terraform.EvalOpFilter, err: 1 error(s) occurred:

* aws_vpc.spark-emr-main: DependencyViolation: The vpc 'vpc-0291df56137e9c5b1' has dependencies and cannot be deleted.
	status code: 400, request id: 41465664-4873-4c92-b8b2-2e0bda3a5f19

Error: Error applying plan:

1 error(s) occurred:

* aws_vpc.spark-emr-main (destroy): 1 error(s) occurred:

* aws_vpc.spark-emr-main: DependencyViolation: The vpc 'vpc-0291df56137e9c5b1' has dependencies and cannot be deleted.
	status code: 400, request id: 41465664-4873-4c92-b8b2-2e0bda3a5f19

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Workaround:
Manually go to the aws web console and delete the VPC. If I do this while is destroying the VPC, Terraform finishes successfully.

Steps to Reproduce

1. Based on Example bootable config at: https://www.terraform.io/docs/providers/aws/r/emr_cluster.html, I've created my own scripts then...
2. terraform init
3. terraform apply
4. terraform destroy -auto-approve

Additional Context

• My setup is almost identical to the sample I mentioned above, except that I don't use keep_job_flow_alive_when_no_steps, instance_group and additional_info.
• I have tried by importing the emr managed security groups into my state, marking them with revoke_rules_on_delete = true before deleting; no luck, still unable to delete the VPC.
• I also have additional security groups configured, all of them use revoke_rules_on_delete = true Still no luck.
• From the error, it seems from Terraform standpoint, there is some resource depending on the VPC that is not being deleted as part of the destruction plan. I guess aws console is well aware of the dependencies and hence is able to destroy them in proper order.
• I see something interesting though. On creation, I can see three associated with my VPC despite I only have one on my scripts, the one I'm explicitly associating to my VPC. I wonder if this is these are the dependencies that prevent the VPC to be deleted.

References

Maybe this is related to:

• EMR Cluster should ignore emr_managed_*_security_group if not defined #9487
• Issue destroying EMR on AWS terraform-provider-aws#3465

[ghost]

This issue has been automatically migrated to hashicorp/terraform-provider-aws#8285 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to hashicorp/terraform-provider-aws#8285.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.