Terraform errantly reports need to update. State file out of sync with deployed resources when no modifications have been made.

[jzampieron]

Terraform Version

Terraform v0.8.8

Affected Resource(s)

Please list the resources as a list, for example:

• azurerm_virtual_machine

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

resource "azurerm_resource_group" "test" {
   name     = "${var.res_group}"
   location = "${var.azurerm_region}"
}

resource "azurerm_virtual_network" "test" {
  name                = "acctvn"
  address_space       = ["10.0.0.0/16"]
  location            = "${var.azurerm_region}"
  resource_group_name = "${azurerm_resource_group.test.name}"
}

resource "azurerm_subnet" "test" {
  name                 = "acctsub"
  resource_group_name  = "${azurerm_resource_group.test.name}"
  virtual_network_name = "${azurerm_virtual_network.test.name}"
  address_prefix       = "10.0.2.0/24"
}

resource "azurerm_public_ip" "test" {
    name                         = "acceptanceTestPublicIp${count.index}"
    location                     = "${var.azurerm_region}"
    resource_group_name          = "${azurerm_resource_group.test.name}"
    public_ip_address_allocation = "dynamic"
    count                        = 2
    tags {
        environment = "${var.instance_env}"
    }
}

resource "azurerm_network_interface" "test" {
  name                = "acctni${count.index}"
  location            = "${var.azurerm_region}"
  resource_group_name = "${azurerm_resource_group.test.name}"
  count               = 2
  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "dynamic"
    public_ip_address_id          = "${element( azurerm_public_ip.test.*.id, count.index )}"
  }
}

resource "azurerm_storage_account" "test" {
  name                = "${var.instance_name}tftestsa"
  resource_group_name = "${azurerm_resource_group.test.name}"
  location            = "${var.azurerm_region}"
  account_type        = "Standard_LRS"

  tags {
    environment = "${var.instance_env}"
  }
}

resource "azurerm_storage_container" "test" {
  name                  = "vhds"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  storage_account_name  = "${azurerm_storage_account.test.name}"
  container_access_type = "private"
}

# Name for the OS disks.
# This is generated w/ a UUID b/c azure can't figure out how to reattach
# _OR_ create if not exists.
# This way, terraform will never delete a disk by accident.
data "template_file" "azurerm_vm_osdisk" {
  template = "acctvm%05d-osdisk-${uuid()}"
}

# Name for the OS disks.
# This is generated w/ a UUID b/c azure can't figure out how to reattach
# _OR_ create if not exists.
# This way, terraform will never delete a disk by accident.
data "template_file" "azurerm_vm_datadisk" {
  template = "acctvm%05d-datadisk-${uuid()}"
}

resource "azurerm_virtual_machine" "test" {
  name                  = "${format( "acctvm%05d", count.index )}"
  location              = "${var.azurerm_region}"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  network_interface_ids = ["${element( azurerm_network_interface.test.*.id, count.index )}"]
  vm_size               = "Standard_A0"
  delete_os_disk_on_termination = true
  count                         = 2
  lifecycle {
     ignore_changes = [ "storage_os_disk", "storage_data_disk" ]
  }

  # Can use: az vm image list-publishers --location eastus2 to help here.
  storage_image_reference {
    publisher = "CoreOS"
    offer     = "CoreOS"
    sku       = "Stable"
    version   = "1235.9.0"
  }

  storage_os_disk {
    name          = "${ format( data.template_file.azurerm_vm_osdisk.rendered, count.index ) }"
    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/${ format( data.template_file.azurerm_vm_osdisk.rendered, count.index ) }.vhd"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }

  # Be really really careful here. If, for some reason Terraform has to nuke
  # and rebuild the VM... you will get a new volume by UUID. This is
  # goofy, but ON PURPOSE b/c Azure API doesn't understand CREATE IF NOT EXISTS,
  # otherwise ATTACH to the image.
  # It's setup so you could manually go back and recover the _old_
  # VHDs and reattach them to the vms using the VM count.index
  # if you really had to.
  # AKA: Efforts are made to Preserve the VHDs, even at the cost of
  # wasting space and/or having dangling VHDs in the storage account.
  storage_data_disk {
    name          = "${ format( data.template_file.azurerm_vm_datadisk.rendered, count.index ) }"
    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/${ format( data.template_file.azurerm_vm_datadisk.rendered, count.index ) }.vhd"
    disk_size_gb  = "512"
    # See: https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/virtualmachines-create-or-update#Anchor_2
    # There isn't an obvious "create if not exists, otherwise attach option. "
    create_option = "Empty"
    lun           = 0
  }

  os_profile {
    computer_name  = "hostname"
    admin_username = "testadmin"
    admin_password = "Password1234!"
    # Base64 encoded ... This is an "Ignition" configuration file for coreos.
    custom_data    = "${base64encode( file( "config.ign" ) )}"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }

  tags {
    environment = "${var.instance_env}"
  }
}

Expected Behavior

Terraform should report that the state file is in sync with the tf files.

Actual Behavior

terraform plan reports the need to update the resources.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply
2. terraform plan -> Note that terraform still tries to report modifications.

Important Factoids

I believe this is related to the base64encode() function not storing the proper value in the state file for the custom_data element.

The state file shows the raw JSON content instead of the base64encoded string shown in the plan output.

It's uncertain what's actually running on the cluster, although I believe it's correct b/c the API call works and the CoreOS configuration is updated.

[jzampieron]

This problem is also happening with the "admin_password" field which is not retrieved from azure API, but exists in the tf files as a real value.

You can set it, but you can't read it back.

[brandontosch]

FYI, I believe the custom_data issue should already be fixed in an upcoming version with this change: #12214

[jzampieron]

@brandontosch If that's the case, I think #12214 needs to also update the azurerm documentation to indicate that the data is base64 encoded internally. As it stands presently, you'd believe that the proper approach is to call "${base64encode( ... )}" as the value.

[brandontosch]

Agreed, and looks like that got done as well, here: #12164
Both pretty recent changes.

[brandontosch]

admin_password though, that may be something to look in to.. it's currently not decorated as sensitive either which seems like it should be (would prevent value from being output in console or logs)

[brandontosch]

Also, noticed your comments in there regarding create_option on the data disk. When managed disk support (#12455) gets added you'll be able to define your disks independently of the VM (using azurerm_managed_disk) and then you can achieve a "create if not exists, otherwise attach" pattern by using the attach create_option (example, can't guarantee that's going to be the exact syntax since it's currently under review, but the pattern should be supported regardless).

[StephenWeatherford]

@jzampieron Note that you can probably work around the admin_password issue by using ssh_keys instead.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.