0.12-rc1 - modules cannot be sourced and installed from git repo

[ilicmilan]

Terraform Version

Terraform v0.12.0-rc1

Terraform Configuration Files

module "chat" {
  source          = "git::ssh://git@bitbucket.bla.com:inf/terraform.git//modules/chat"
}

Debug Output

2019/05/09 11:31:59 [DEBUG] Module installer: begin chat
2019/05/09 11:31:59 [TRACE] ModuleInstaller: chat is not yet installed
2019/05/09 11:31:59 [TRACE] ModuleInstaller: cleaning directory .terraform/modules/chat prior to install of chat
2019/05/09 11:31:59 [TRACE] ModuleInstaller: chat address "git::ssh://git@bitbucket.bla.com:inf/terraform.git//modules/chat" will be handled by go-getter
2019/05/09 11:31:59 [DEBUG] will download "git::ssh://git@bitbucket.bla.com:inf/terraform.git" to .terraform/modules/chat
2019/05/09 11:31:59 [TRACE] fetching "git::ssh://git@bitbucket.bla.com:inf/terraform.git" to ".terraform/modules/chat"
Downloading git::ssh://git@bitbucket.bla.com:inf/terraform.git for chat...
2019/05/09 11:31:59 [TRACE] modsdir: writing modules manifest to .terraform/modules/modules.json

Error: Failed to download module

Error attempting to download module "chat" (main.tf:157) source code from
"git::ssh://git@bitbucket.bla.com:inf/terraform.git":
<nil>

Expected Behavior

terraform init should succeed

Actual Behavior

terraform init has failed

Steps to Reproduce

Use a Git repo and reference it as specified above
terraform init

Additional Context

These git references work in 0.11.x

References

• In 0.12, modules can no longer be installed from local git repositories at relative paths #21107

[dirkcjelli]

I get the same error, but for some reason, I get it twice in a row, command exits after 75 seconds:

2019/05/09 17:13:38 [INFO] Terraform version: 0.12.0 rc1
2019/05/09 17:13:38 [INFO] Go runtime version: go1.12.1
2019/05/09 17:13:38 [INFO] CLI args: []string{"/Users/dirkcjelli/bin/terraform", "init"}
2019/05/09 17:13:38 [DEBUG] Attempting to open CLI config file: /Users/dirkcjelli/.terraformrc
2019/05/09 17:13:38 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/05/09 17:13:38 [INFO] CLI command args: []string{"init"}
Initializing modules...
2019/05/09 17:13:38 [DEBUG] Module installer: begin vpc_base
2019/05/09 17:13:38 [TRACE] ModuleInstaller: vpc_base is not yet installed
2019/05/09 17:13:38 [TRACE] ModuleInstaller: cleaning directory .terraform/modules/vpc_base prior to install of vpc_base
2019/05/09 17:13:38 [DEBUG] will download "git::ssh://git@mygitlab:2424/terraform/terraform-aws/module-vpc.git?ref=0.12" to .terraform/modules/vpc_base
Downloading git::ssh://git@mygitlab:2424/terraform/terraform-aws/module-vpc.git?ref=0.12 for vpc_base...

Error: Failed to download module

Error attempting to download module "vpc_base" (main.tf:1) source code from
"git::ssh://git@mygitlab:2424/terraform/terraform-aws/module-vpc.git?ref=0.12":
<nil>


Error: Failed to download module

Error attempting to download module "vpc_base" (main.tf:1) source code from
"git::ssh://git@mygitlab:2424/terraform/terraform-aws/module-vpc.git?ref=0.12":
<nil>

75.27 real         0.12 user         0.12 sys

[apparentlymart]

Thanks for reporting this @ilicmilan, and @dirkcjelli for the additional context.

This is interesting because I don't believe we changed the install-from-git code in Terraform v0.12, but possibly changes to the upstream library we use for module installation have inadvertently changed the behavior here. We'll investigate what's changed here and get it fixed.

Thanks again!

[richardj-bsquare]

I also get this, with the line repeated for each reference to the module... I made minimal changes to my terraform such that it would work with 0.11.13 and 0.12. Terraform 0.11.13 managed to checkout via my SSH agent just fine, but get the above with 0.11-beta2 and 0.11-rc1. From my investigation, it worked fine with SSH references to GitHub, but not our cloud-based bitbucket server... My temporary workaround was to move some code to GitHub...

I thought it might be defaulting to port 22 for SSH?

[jocelynthode]

I have the same problem mentionned above with something like:
source = "git::ssh://git@foo.net:7999/terraform-modules/xxx/yyy.git?ref=master"

It asks for a password. It seems to completely ignore the specified ssh port and defaults to 22.

When modifying the ~/.ssh/config with something like:

Host foo.net
    HostName foo.net
    User git
    Port 7999

I then get the aforementioned issue.

The workaround is to modify the code as following:

• Remove the port from the code source = "git::ssh://git@foo.net/terraform-modules/xxx/yyy.git?ref=master"
• Add to your ssh config the following:

Host foo.net
    HostName foo.net
    User git
    Port 7999

It should then initialize fine.

[apparentlymart]

It seems that this was broken by hashicorp/go-getter#129, because all of the URLs shared here are matching the heuristic it tries to use to detect "SCP-style" syntax.

As a consequence, ssh://git@mygitlab:2424/terraform/terraform-aws/module-vpc.git is being rewritten to ssh://git@mygitlab/2424/terraform/terraform-aws/module-vpc.git, which of course then fails to actually work.

Because the SCP-style URLs have now been supported for several releases, we unfortunately cannot remove that behavior to fix this, so instead I'm going to investigate whether we can adjust the heuristic to be more conservative and not break explicit port numbers.

Unfortunately, the SCP-style syntax support creates an ambiguity, so anything we do here is going to be heuristic: git@mygitlab:2424/terraform/terraform-aws/module-vpc.git could be interpreted as either of the following:

• Connect to mygitlab on port 2424 and retrieve /terraform/terraform-aws/module-vpc.git
• Connect to mygitlab on port 22 and retrieve 2424/terraform/terraform-aws/module-vpc.git

This ambiguity presumably also exists in Git itself, because Git supports both forms, so I'm going to start by tracking down the relevant logic in Git and see if we can mimic its heuristic more accurately.

[dirkcjelli]

@jocelynthode Thanks for the hint with the ssh-config, works like a charm 👍

[apparentlymart]

With #21293 merged, this is now fixed in the master branch. The fix will be included in the forthcoming v0.12.0 final release. Sorry for the accidental regression, everyone!

Thanks for sharing that workaround, @jocelynthode! That's a good way to get around this if you'd like to do other testing with v0.12.0-rc1, though you shouldn't need it any more after v0.12.0 final.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.