Automatically preserve functionally-equivalent configuration values during refresh

[apparentlymart]

In Terraform's model, there are conceptually two different ways to interpret a difference between the configuration and a state. If we assume that it's the state that's changed while the config remains the same, those cases are:

• Drift: something has changed outside of Terraform such that the remote object state is meaningfully different than what was configured. Terraform should plan to repair this by setting it back to match the configuration.
• Normalization: there are multiple ways to express the same idea and the user chose a different formatting in the config than the remote system considers "canonical". In this case, Terraform should ideally do nothing, because the difference is not meaningful.

(If it's the configuration that changed rather than the state then the exact definitions of the above would be different, but the same distinction still exists.)

In the original SDK (helper/schema) we modeled this difference using DiffSuppressFunc on a per-attribute basis, allowing the provider to decide for each given change whether it is "Drift" (return false) or "Normalization" (return true).

Unfortunately, DiffSuppressFunc is considered only during planning, which doesn't cover the common case where the Read function updates the state to match the value returned by the remote API, thus making a normalization change effective in the state. On a subsequent plan, that attribute itself may not show up as having a diff (because DiffSuppressFunc hopefully returned true) but any other resource whose configuration depends on that value will now show a diff, because that downstream attribute may not even belong to the same provider and thus can't possibly know the logic for deciding Drift vs. Normalization in the general case.

Under the general theme of having the SDK do "the right thing" by default, it'd be nice if the SDK would apply DiffSuppressFunc (or whatever similar features evolve from it) also to the result of Read, so if the provider sets a value that is different only in the normalization sense, the SDK would discard that new value and retain the original value as the user wrote it.

---

The idea that the configuration takes priority in the case of any disagreement is a fundamental assumption in Terraform Core because without it interpolation cannot function reliably: any value derived from that result could otherwise potentially change in response to normalization, causing apply failures or diffs that take one or more additional runs to converge. This is one reason why we implemented DiffSuppressFunc even though similar use-cases were already covered by StateFunc.

(From a more subjective standpoint, I also think it's the best user experience: if I write "Foo" in the configuration and I interpolate that value somewhere else, it'd be pretty surprising to see that evaluate as "foo".)

[bflad]

The next version of terraform-plugin-sdk/v2, unreleased at the moment, but slated to be v2.11.0 adds an additional DiffSuppressOnRefresh field to helper/schema.Schema. When enabled, this checks the DiffSuppressFunc during refresh. If the planned value matches the old state value via the DiffSuppressFunc, the old value will be preserved (or put another way, the planned value will be ignored). This should help mitigate cases like these where the DiffSuppressFunc implementations were likely put in place with the expectation that this would already happen in the SDK, however it previously did not.

We expect DiffSuppressOnRefresh to be generally safe to apply in most cases where DiffSuppressFunc is already being used, however it is not enabled by default for backwards compatibility.

---

For tracking of similar functionality in terraform-plugin-framework, refer to hashicorp/terraform-plugin-framework#70.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.