terraform plan reports "planned for absence but config wants existence" when provider returns empty Set

[ewbankkit]

Terraform Version

% terraform version
Terraform v1.0.6
on darwin_amd64

Description

A protocol v6 resource type declares (using terraform-plugin-framework) an attribute tags of type:

"tags": {
	Attributes: types.SetNestedAttributes(
		map[string]tfsdk.Attribute{
			"key": {
				Type:        types.StringType,
				Required:    true,
			},
			"value": {
				Type:     types.StringType,
				Required: true,
			},
		},
	),
	Optional: true,
},

My configuration has no tags defined:

resource "cloud_service_thing" "test" {}

On terraform refresh the underlying cloud provider API, and hence the provider, returns an empty array for tags:

2021-09-14T10:08:48.938-0400 [WARN]  Provider "registry.terraform.io/hashicorp/cloud" produced an unexpected new value for cloud_service_thing.test during refresh.
      - .tags: was null, but now cty.SetValEmpty(cty.Object(map[string]cty.Type{"key":cty.String, "value":cty.String}))

and Terraform correctly shows a diff:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # cloud_service_thing.test will be updated in-place
  ~ resource "cloud_service_thing" "test" {
        id                  = "606d124b-cfcc-40cb-b4b5-f37bf5d70c85"
      - tags                = [
        ]
        # (8 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

I change my configuration to try and suppress the diff and match the cloud provider API's result:

resource "cloud_service_thing" "test" {
  tags = []
}

On terraform refresh I now get an error:

2021-09-14T10:11:34.877-0400 [WARN]  Provider "registry.terraform.io/hashicorp/cloud" produced an unexpected new value for cloud_service_thing.test during refresh.
      - .tags: was null, but now cty.SetValEmpty(cty.Object(map[string]cty.Type{"key":cty.String, "value":cty.String}))
2021-09-14T10:11:34.899-0400 [INFO]  backend/local: plan operation completed
╷
│ Error: Provider produced invalid plan
│ 
│ Provider "registry.terraform.io/hashicorp/cloud" planned an invalid value for cloud_service_thing.test.tags: planned for absence but config wants existence.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

I would expect Terraform to show no diff when both the configuration specifies an empty set and the provider/underlying cloud provider API returns an empty set.

[apparentlymart]

Hi @ewbankkit! Thanks for reporting this.

I'm still getting familiar with the new framework structures so possibly I'm misunderstanding the schema here, but it looks like this tags attribute is set to Optional and not Computed, which means that Terraform expects a null (unset) in the configuration to stay unset in the planned new value.

Replacing a null value with an empty set here is, from Terraform Core's perspective, the provider setting a default value for this argument, which we currently represent as the combination of Optional+Computed.

Given that, you might be able to get the result you expected by marking this attribute also as Computed in addition to your existing Optional.

[jbardin]

This type is issue is something that has always been kind of a pain point for providers, but was masked by old protocol, because when we call ReadResource the remote service unfortunately does not follow the same protocol constraints as Terraform.

What's happening here is that the resource is planning the change correctly, and honoring the configuration for tags, but later during ReadResource the remote service is returning an empty value for tags rather than null. It may be possible in some cases for the provider to have enough knowledge and ability to normalize these values, but technically the ReadResource call could be reporting "drift" of some sort which does not match configuration, and the response would still be valid.

So adding the tags to the config should technically be valid here, and should remove the "drift" report. I think the oddity here is how tags got to be null again during the plan validation, since it should be cty.SetValEmpty from the refresh call which matches the configuration.

The error message in this case I believe is one where I already tried to differentiate between a terraform bug and provider, but there was not enough information to do so, so stuck with This is a bug in the provider since it's the most common path.

[apparentlymart]

Oh hmm yes, I was focusing on the final error message as included in the summary and didn't spot the context that the first weird error was from the ReadResource step.

In that case, I think this might be an example of what I was discussing in hashicorp/terraform-plugin-framework#70, where I think ideally the SDK would have a way for a provider developer to define what makes two values be considered equal and then use that both for planning and for refreshing, so we always stay consistent with the configuration.

In the meantime though, indeed it seems like it would work either to do what I said above and thus make the empty set be a "default" for this argument, or to go the other way and have ReadResource treat an empty set of tags as a null value for this attribute. In other words, to make the planning behavior consistent with the refreshing behavior in either one direction or the other.

[ewbankkit]

@apparentlymart This does seem to be an example of the behavior discussed in hashicorp/terraform-plugin-framework#70; The absence of an array (set or list) and an empty array are semantically equivalent at the cloud provider API level, at least for this particular attribute.
For other attributes where an explicit default value has been documented we are setting the attribute's properties as Optional and Computed but for this example, since no default value has been documented we are using just Optional.

@jbardin Yes,

the oddity here is how tags got to be null again during the plan validation

Is this something that is likely to be happening provider-side?

[jbardin]

@ewbankkit, I'm going to come up with a repro to verify that and report back here. This is in an area where terraform blames the provider because it's most likely, but not always guaranteed.

[jbardin]

@ewbankkit: Confirmed the error is in Terraform. While it may be nice if the SDK provided some way to help providers normalize nil or empty values to prevent needing tags = [] in the config, this particular error is caused by an invalid ProposedNewState generated from the nested attribute types.

[ewbankkit]

@jbardin terraform-plugin-framework has a powerful plan modification extensibility facility that allows normalization to occur, although if this is universal behavior for this cloud provider's API I should be able to address the normalization at a lower, marshalling layer in the provider.
Having the diff detected correctly in the Terraform CLI will help inform the way forward in the provider.
Thanks.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.