Terraform new workers post-upgrade, tfstate problem

[djtaylor]

What keywords did you search in tectonic-installer issues before filing this one?

tfstate, tfvars, inconsistency, upgrade, metal, worker

Is this a BUG REPORT or FEATURE REQUEST?

FEATURE REQUEST

We have an existing cluster that was set up with Terraform on bare-metal from the 1.7.5 installer. We have since upgraded to 1.96 (including intermediate releases) and are now looking to add additional worker nodes. However, we are running into issues when trying to reconcile the tfstate file from the initial installation with the current cluster.

I have looked at the documentation at https://coreos.com/tectonic/docs/latest/troubleshooting/tectonic-upgrade.html, but I believe this assumes you are running terraform to add new worker nodes with the same installer version as you are currently running.

I have tried copying over the old tfstate file from 1.7.5 and using terraform refresh and terraform plan, but am getting a number of concerning warnings about forcing the recreation of new resources. When I generate a new state file by running terraform refresh with an empty state file, I don't get any warnings, but based on the output of the command, it looks like it is going to overwrite/replace existing items which may break our infrastructure.

We would like some more comprehensive documentation on how to scale bare-metal workers with terraform using a newer version that what you initially provisioned with.

Versions

Tectonic version (release or commit hash):

Provisioned with: 1.7.5-tectonic.1
Currently running: 1.9.6-tectonic.1

Terraform version (terraform version):

v0.10.7

Platform (aws|azure|openstack|metal|vmware):

metal

What happened?

We are trying to add new metal workers to the cluster with terraform, but are on a newer version of the installer/Tectonic since we initially deployed. I am seeing resources being generated in `terraform plan` that already exist in the cluster.

What you expected to happen?

`terraform refresh` and/or `terraform/plan` should be able to accurately get the state of the current cluster on a newer version of Tectonic.

How to reproduce it (as minimally and precisely as possible)?

Bootstrap a cluster with an older version of installer. Download the latest release of the installer and try to generate a new tfstate file that accurately represents the cluster for a run of `terraform apply`.

Anything else we need to know?

enter text here

References

None that I can find. This is a mix between a Terraform and Tectonic question.

Feature Request

Environment

We are running on bare-metal.

Desired Feature

More robust documentation for using the Tectonic installer after an upgrade (from using an older version of the Tectonic installer).

Other Information

Thats all I have to say about that.