Terraform ignores dependencies/order of operations for resources that are modified in place

[vladimir-avinkin]

Terraform Version

terraform version
Terraform v1.2.9
on linux_amd64
+ provider registry.terraform.io/hashicorp/hashicups v0.3.1
+ provider registry.terraform.io/hashicorp/null v3.1.1

Terraform Configuration Files

https://github.com/vladimir-avinkin/terraform-dependency-order-repro

Debug Output

https://gist.github.com/vladimir-avinkin/f948eb7d4917e2c590d4d3172f70e144

Expected Behavior

Terraform should respect the order of operations for dependent resources, in this case, null_resource.null[1] should have been destroyed AFTER modifying hashicups_order.cup, not before

Actual Behavior

Despite the explicit and explicit dependency terraform destroyed null_resource.null[1] before modifying hashicups_order.cup

Steps to Reproduce

Clone the repository, ensure that you have the required tools that are listed in README.md, then run:

docker-compose up -d
# make sure you wait for the service to be up before the next command
curl -X POST localhost:19090/signup -d '{"username":"education", "password":"test123"}'
terraform init
# after that command terraform should successfully create three resources 
terraform apply
# now run apply with fewer dependencies
terraform apply -var 'resource_count=1'

In that last apply you will see that terraform deletes null_resource.null[1] before modifying hashicups_order.cups despite
hashicups_order.cups implicit and explicit dependencies

Additional Context

No response

References

• I believe the same bug is described in that issue Terraform deletes/modifies resource and its dependency simultaneously #23169

[jbardin]

Hi @vladimir-avinkin,

The example given is not missing any dependencies, the order of operations is well-defined and is being executing as they are defined. If it helps, think about the situation where null_resource.null[1] were being replaced, in which case hashicups_order.cups could not be updated until after null_resource.null[1] was both destroyed and re-created. The case where the resource is only being destroyed must follow the same overall order of operations. If you want to invert this ordering, you can create null_resource.null using create_before_destroy, which will ensure the create and update operations happen before the destroy operations.

Since this does not represent a bug in Terraform, I'm going to close the issue. If you have more questions, feel free to continue the conversation in the original forum thread.

Thanks!

[vladimir-avinkin]

@jbardin Yes, you are right, sorry, I tested it with create_before_destroy initially but somehow missed it. It's still hard to wrap my head around that behavior and it seems like that's actually two separate operations, update + delete and update + delete + create + update

If we would do that this way there would be no need to use create_before_destroy.

For my particular case that seems to work, but I think having the ability to support two instances of resources at once is provider specific, right? So I still tend to think that it's the problem with core terraform functionality if you can't set the order of operations without creating two instances of resources at once.

[jbardin]

The overall order of resource actions must be consistent, regardless of whether the resource is being replaced, only deleted, or only updated. This is especially important in large configurations where it would be very confusing for users if removing vs replacing a resource subtly changed the overall order of operations causing other failures. Terraform must pick some order to use by default, and none of the choices can work for every possible configuration, so we always start from delete-then-create.

Also, once you build out more complex graphs of dependencies, there are very few topological orders which can create a valid dependency graph at all. It's tempting to assume that the simple case where you want to swap the order of operations is valid, but once that is combined into arbitrarily large graphs of changes, cycles appear making the proposed order invalid. I made some notes to help explain the basics here if you're interested: https://github.com/hashicorp/terraform/blob/main/docs/destroying.md.

[vladimir-avinkin]

Thank you for spending your time explaining that to me, I can't honestly say that I understood this document completely, so feel free to ignore my ramblings if I missed something entirely :)

But again, as I said the problem with your scenario

If it helps, think about the situation where null_resource.null[1] were being replaced, in which case hashicups_order.cups could not be updated until after null_resource.null[1] was both destroyed and re-created.

Is not actually the order itself, but because we treat replace (delete+create) + update as three operations, when it's actually 4 update + replace (delete +create) + update if you think about it.

I can't see how the order of operations would break at all in any scenario if we just update the dependent resource twice.

[jbardin]

You cannot update the hashicups_order before the null_resource create, because the hashicups_order depends on the null_resource . Perhaps you were thinking of something like delete -> update -> create -> update, but that still leaves us with the problem that we need a single plan of how we are going to execute the hashicups_order, but there are 2 different update actions here for the same resource instance.

Besides the fact that the architecture of Terraform is not designed around being able to handle multiple updates like this, we also have to contend with the fact that providers (and hence the implementations of each resource) would never expect to have multiple updates to a single resource plan or apply operation.

Say for instance the dependency was used in a required attribute on the hashicups_order. The application of that first update plan would be invalid as we are breaking the invariant of giving the provider that required attribute from the configuration -- only once the dependency has been fully resolved do we have the necessary information for the update. So even if Terraform were re-imagined to have multiple actions planned per resource, in this instance we still could not insert the preliminary update because it would cause the provider to fail.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.