Terraform silently exits on windows using absolute plan path

[avvi00]

Issue Summary

I successfully run terraform plan and save the plan to a file. When I run terraform apply - nothing happens - it immediately exits with no error and no crash log. Exit code is 1 though.

Terraform Version

Windows AMD64

Terraform v0.12.25
+ provider.aws v2.63.0

Terraform Configuration Files

Fairly complex config with multiple modules is attached. File is encrypted with hashicorp public key

tf-github-issue.gpg.zip

Debug Output

2020/05/29 10:45:01 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
2020/05/29 10:45:01 [INFO] Terraform version: 0.12.25  
2020/05/29 10:45:01 [INFO] Go runtime version: go1.12.13
2020/05/29 10:45:01 [INFO] CLI args: []string{"C:\\ProgramData\\Simplogy\\Simpology.DevOps.Terraform\\bin\\0.12.25\\terraform.exe", "apply", "d:\\temp\\tf-github-issue\\shared-infrastructure\\layers\\000account\\out\\000account.plan"}
2020/05/29 10:45:01 [DEBUG] Attempting to open CLI config file: C:\Users\Avner\AppData\Roaming\terraform.rc
2020/05/29 10:45:01 Loading CLI configuration from C:\Users\Avner\AppData\Roaming\terraform.rc
2020/05/29 10:45:01 [INFO] CLI command args: []string{"apply", "d:\\temp\\tf-github-issue\\shared-infrastructure\\layers\\000account\\out\\000account.plan"}

(Note that terraform.rc is empty. I get the same result with no terraform.rc. )
Full output from each command is attached. File is encrypted with hashicorp public key

tf-github-issue.gpg.zip

Crash Output

No crash.log produced.

Expected Behavior

I expect terraform apply to run a plan file that was successfully generated.

Actual Behavior

The command exits without applying the plan. Debug output from the command is shown above.
The exit code of terraform is 1.

Steps to Reproduce

Below are the steps I'm running:

terraform init -var "aws_account_alias=abcdef" -var "aws_account_id=12345678" -var "layer=000account" -backend-config="key=12345678/000account.tfstate" -reconfigure

terraform get -update=true

terraform plan -var "aws_account_alias=abcdef" -var "aws_account_id=12345678" -var "layer=000account" -out d:\temp\tf-github-issue\shared-infrastructure\layers\000account\out\000account.plan -var-file=d:\temp\tf-github-issue\shared-infrastructure\layers\000account\env\abcdef.tfvars

terraform apply d:\temp\tf-github-issue\shared-infrastructure\layers\000account\out\000account.plan

Workaround

I can get terraform apply to run by not using the plan file and just giving it the same parameters as plan.

Additional Context

Not sure if relevant but I have been upgrading my modules from v11 to v12 and referencing v12 external modules.

[danieldreier]

Thanks for reporting this! What a puzzling issue. Do you have a Linux or OS X system you can test this on, to see if this is specific to running on windows?

[avvi00]

Hi @danieldreier - yep can try from a linux box. Will post back.

[avvi00]

@danieldreier thanks - I've tried on a Linux machine and it works as expected. Therefore it's an issue with the windows binary. Anything else you need?

[danieldreier]

@avvi00 I just need to find the time to reproduce it! I've got what I need from you, thank you - I'll let you know when I've reproduced it. I appreciate you reporting this and making all the source available to us.

[avvi00]

Hi @danieldreier no worries and thank you for taking the time to investigate. It's a strange issue but I believe it's linked to the upgrade of tf 11 to tf 12. specifically having state in tf 11 format, and producing a plan in tf 12.

Now that I've used the workaround to upgrade, I don't see the issue anymore for all subsequent resource creation (state and plan in tf 12).

[danieldreier]

@avvi00 I don't quite understand how tf 0.11 state factors in here - the reproduction instructions you gave me don't account for a step where 0.11 state would be created. In reproducing this, where is the 0.11-format state supposed to come from?

[avvi00]

@danieldreier the 0.11 state comes from existing infrastructure built. So there was a whole lot of stuff which was previously built on 0.11 - then I undertook the upgrade to 0.12 and encountered the issues. It was implied there was existing infrastructure with my comment about upgrading, but should have been made clearer.

I've attached state prior to the upgrade for your investigation.
000account.gpg.zip

[danieldreier]

@avvi00 can you help me understand the impact here? If I'm understanding right, you have already worked around this and are no longer impacted for subsequent resource creation. Is that right?

I'm just trying to triage because fixing an upgrade from 0.11 to 0.12 is unlikely to happen at this point, given that we are not planning any more 0.12 releases and there is no direct upgrade path from 0.11 to 0.13.

Also, you should remove the state file you uploaded: this almost certainly contains AWS account credentials that we should not have access to.

[avvi00]

@danieldreier you understand correctly. There is no further impact to me, but only to others who might do the upgrade from 0.11 to 0.12. I'm sure there is still a lot of 0.11 out there e.g. rackspace who we work with have all their IaC in terraform 0.11. For them to get to 0.13 they need to first upgrade to 0.12. There is a workaround though - so I'd completely understand if you close this.

I checked the files for credentials / sensitive data before uploading, but I've asked for deletion just in case.

[avvi00]

Hi @danieldreier

Good news - the issue cropped up again for me today, and I've got the exact steps to replicate. It's got nothing to do with existing state etc.. forget about all the above 😃

The problem appears to be when using an absolute path for the apply command:

• create the file: c:\src\tfwindowstest\main.tf

provider "aws" {
  version             = "~> 2.64"
  region              = "ap-southeast-2"
}

resource "random_id" "this" {
  byte_length = "8"
}

resource "aws_s3_bucket" "testbucket" { 
  bucket = "test-bucket123-${random_id.this.hex}"
}

• run plan

terraform plan -out "C:\src\tfwindowstest\tfwindowstest.plan"

• run apply with absolute path

terraform apply "C:\src\tfwindowstest\tfwindowstest.plan"

• crash
• run apply with relative path (cd to c:\src\tfwindowstest)

terraform apply .\tfwindowstest.plan

• works

Tested this on v0.12.26

[danieldreier]

@avvi00 this is great! This is so much simpler, thank you. Thanks to you investigating further, I've been able to simplify this test case even further:

example.tf

resource "null_resource" "test" {}

It looks to me like you are correct, and the critical issue is the use of an absolute path, which causes terraform to exit with exit code 1 early in the process. With LOG_LEVEL=TRACE, I get:

PS Z:\src\terraform-issue-reproductions\25068> terraform-0.12.26.exe apply "Z:\src\terraform-issue-reproductions\25068\testplan.plan"
2020/06/11 11:06:09 [INFO] Terraform version: 0.12.26
2020/06/11 11:06:09 [INFO] Go runtime version: go1.12.13
2020/06/11 11:06:09 [INFO] CLI args: []string{"C:\\Users\\daniel\\bin\\terraform-0.12.26.exe", "apply", "Z:\\src\\terraform-issue-reproductions\\25068\\testplan.plan"}
2020/06/11 11:06:09 [DEBUG] Attempting to open CLI config file: C:\Users\daniel\AppData\Roaming\terraform.rc
2020/06/11 11:06:09 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/06/11 11:06:09 [INFO] CLI command args: []string{"apply", "Z:\\src\\terraform-issue-reproductions\\25068\\testplan.plan"}

I have reproduced this on terraform 0.12.26 and 0.13.0 beta 1 on Windows 10. I will add this to our internal backlog as a valid, reproduced issue. Thank you!

[avvi00]

Glad I could help out @danieldreier ! Thanks to you and the core team for a great product.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.