Problems using CustomizeDiff with calculated fields

[yoink00]

SDK version

{
  "Path": "github.com/hashicorp/terraform-plugin-sdk/v2",
  "Version": "v2.0.1"
}

Relevant provider source code

		CustomizeDiff: func(ctx context.Context, d *schema.ResourceDiff, meta interface{}) error {
			old, new := d.GetChange("network_interface")

			oldList := old.([]interface{})
			newList := new.([]interface{})

			// If the new list is shorter than the old list we
			// want to force new.
			if len(newList) < len(oldList) {
				d.ForceNew("network_interface")
				return nil
			}

			for i, oi := range oldList {
				oiB := oi.(map[string]interface{})
				niB := newList[i].(map[string]interface{})

				if oiB["type"] != niB["type"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.type", i))
				}
				if oiB["ip_address_family"] != niB["ip_address_family"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.ip_address_family", i))
				}
				if oiB["network"] != niB["network"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.network", i))
				}
			}

			return nil
		},

Terraform Configuration Files

Initial Configuration

resource "upcloud_server" "my-server" {
  zone     = "fi-hel1"
  hostname = "debian.example.com"
  plan     = "2xCPU-4GB"

  storage_devices {
    action = "create"
    size   = 10
    tier   = "maxiops"
  }

  network_interface {
    type = "utility"

  }

  network_interface {
    type = "private"

    network = upcloud_network.test_network_1.id
  }

}

resource "upcloud_network" "test_network_1" {
  name = "test_network_1"
  zone = "fi-hel1"

  ip_network {
    address            = "10.0.34.0/24"
    dhcp               = true
    dhcp_default_route = false
    family             = "IPv4"
    gateway            = "10.0.34.1"
  }
}

Updated Configuration

resource "upcloud_server" "my-server" {
  zone     = "fi-hel1"
  hostname = "debian.example.com"
  plan     = "2xCPU-4GB"

  storage_devices {
    action = "create"
    size   = 10
    tier   = "maxiops"
  }

  network_interface {
    type = "utility"

  }

  network_interface {
    type = "private"

    network = upcloud_network.test_network_11.id
  }

}

resource "upcloud_network" "test_network_1" {
  name = "test_network_1"
  zone = "fi-hel1"

  ip_network {
    address            = "10.0.34.0/24"
    dhcp               = true
    dhcp_default_route = false
    family             = "IPv4"
    gateway            = "10.0.34.1"
  }
}

resource "upcloud_network" "test_network_11" {
  name = "test_network_11"
  zone = "fi-hel1"

  ip_network {
    address            = "10.0.24.0/24"
    dhcp               = true
    dhcp_default_route = false
    family             = "IPv4"
    gateway            = "10.0.24.1"
  }
}

Debug Output

Crash Log: https://gist.github.com/yoink00/09ae6424b6d0a468aca49dba10935259

Expected Behavior

Changing the network attribute in the existing network_interface list element should force a new upcloud_server resource.

The CustomizeDiff function is attempting to detect a few scenarios where we know we need to re-create the resource: a) when we've removed an item from the list b) any of the items in the list have certain fields changed or c) the order of the list has changed.

This code mostly works in manual and automated tests. The network field is the ID of a separate upcloud_network resource. If the creation of that resource happens during the same plan/apply as changing the value of network then the provider crashes (i.e. the network ID is not yet known). If the new ID is known (i.e. the new network was created during a previous apply) it does not crash.

Actual Behavior

The provider crashes. The crash appears to be happening in line 796 of grpc_provider.go here (at if v.NewExtra != nil {):

	newExtra := map[string]interface{}{}

	for k, v := range diff.Attributes {
		if v.NewExtra != nil {
			newExtra[k] = v.NewExtra
		}
	}
	privateMap[newExtraKey] = newExtra

Sticking in a debug Printf we can see that the value of network is nil:

2020/08/23 21:17:56 [DEBUG]: diff.Attributes: map[string]*terraform.ResourceAttrDiff{"network_interface.1.network":<nil>}

Steps to Reproduce

1. terraform apply ; # with initial configuration
2. terraform apply ; # with configuration updated to add new network and use ID in exsiting server

References

Forum discussion [https://discuss.hashicorp.com/t/problems-using-customizediff-with-calculated-fields/13251/3]

[paddycarver]

Does v2.4.2 resolve this issue?

[bflad]

It appears this should be resolved already. If that is not the case, please open a new issue. Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.