0.10.6 regression: can no longer source modules from parent directory

[kaii-zen]

Terraform Version

$ terraform -v
Terraform v0.10.6

Terraform Configuration Files

./main.tf:

module "bug" {
  source = "./bug.tgz//foo"
}

output "out" {
  value = "${module.bug.out}"
}

./bug.tgz//foo/foo.tf

module "bar" {
  source = "../bar"
}

output "out" {
  value = "${module.bar.out}"
}

./bug.tgz//bar/bar.tf

output "out" {
  value = "baz"
}

(tarball here)

Expected Behavior

With Terraform 0.10.5:

bash-4.4$ brew switch terraform 0.10.5
Cleaning /usr/local/Cellar/terraform/0.10.0
Cleaning /usr/local/Cellar/terraform/0.10.1
Cleaning /usr/local/Cellar/terraform/0.10.2
Cleaning /usr/local/Cellar/terraform/0.10.3
Cleaning /usr/local/Cellar/terraform/0.10.4
Cleaning /usr/local/Cellar/terraform/0.10.5
Cleaning /usr/local/Cellar/terraform/0.10.6
Cleaning /usr/local/Cellar/terraform/0.9.11
Cleaning /usr/local/Cellar/terraform/0.9.4
Cleaning /usr/local/Cellar/terraform/0.9.5
Cleaning /usr/local/Cellar/terraform/0.9.6
Cleaning /usr/local/Cellar/terraform/0.9.8
Cleaning /usr/local/Cellar/terraform/0.9.9
2 links created for /usr/local/Cellar/terraform/0.10.5
bash-4.4$ terraform version
Terraform v0.10.5

Your version of Terraform is out of date! The latest version
is 0.10.6. You can update by downloading from www.terraform.io
bash-4.4$ tar xzf issue.tgz
bash-4.4$ cd issue
bash-4.4$ ls -l
total 16
drwxr-xr-x  3 kreisys  staff  102 20 Sep 14:31 bar
-rw-r--r--  1 kreisys  staff  272 20 Sep 14:31 bug.tgz
drwxr-xr-x  3 kreisys  staff  102 20 Sep 13:38 foo
-rw-r--r--  1 kreisys  staff   93 20 Sep 14:17 main.tf
bash-4.4$ terraform init
Downloading modules...
Get: file:///Users/kreisys/test/issue/bug.tgz
Get: file:///Users/kreisys/test/issue/.terraform/modules/b4d22ef7134539b7ce3b264a60b62ecd/bar

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
bash-4.4$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.
bash-4.4$ terraform apply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

out = baz

Actual Behavior

With Terraform 0.10.6:

bash-4.4$ brew switch terraform 0.10.6
Cleaning /usr/local/Cellar/terraform/0.10.0
Cleaning /usr/local/Cellar/terraform/0.10.1
Cleaning /usr/local/Cellar/terraform/0.10.2
Cleaning /usr/local/Cellar/terraform/0.10.3
Cleaning /usr/local/Cellar/terraform/0.10.4
Cleaning /usr/local/Cellar/terraform/0.10.5
Cleaning /usr/local/Cellar/terraform/0.10.6
Cleaning /usr/local/Cellar/terraform/0.9.11
Cleaning /usr/local/Cellar/terraform/0.9.4
Cleaning /usr/local/Cellar/terraform/0.9.5
Cleaning /usr/local/Cellar/terraform/0.9.6
Cleaning /usr/local/Cellar/terraform/0.9.8
Cleaning /usr/local/Cellar/terraform/0.9.9
2 links created for /usr/local/Cellar/terraform/0.10.6
bash-4.4$ terraform version
Terraform v0.10.6

bash-4.4$ tar xzf issue.tgz
bash-4.4$ cd issue
bash-4.4$ terraform init
Downloading modules...
Get: file:///Users/kreisys/test/issue/bug.tgz//foo
Error downloading modules: Error loading modules: module bar: invalid source string: ../bar

Steps to Reproduce

With Terraform 0.10.6:

1. Download issue.tgz
2. tar xzf issue.tgz
3. cd issue
4. terraform init

Important Factoids

This happens with GitHub-sourced modules as well and I assume with any source that utilizes that double slash (//) syntax to determine the root of a module collection. I used a tgz file in the repro because this doesn't happen when using the local syntax source = "./foo". Interestingly, the same error occurs if I try to use the double slash syntax with a local path: source = ".//foo"; but I'm not sure if that's even supposed to work in the first place.

[jbardin]

Hi @kreisys,

Sorry about the regression here. It looks like this wasn't intended to work this way, but of course it did work this way so there are now other use cases to consider ;)

For reference:

The syntax "./bug.tgz//foo" instructs the getter to unpack the bug.tgz tarball and use the foo directory as the module source. Conceptually, there is no parent directory above the foo directory for "../bar" to be resolved. The legacy codepath however was unpacking the tarball in place, keeping the full directory structure, but redirecting requests for the bug module to the foo subdirectory.

[ze42]

So, this is a regression, and should either be fixed, or be specified in Changelogs.

Will become an other pain on the disk until modules can use a single directory .terraform/modules/xxx for the same source path.

[navaati]

Hi.
I'm hitting this bug too, and hard. We've got a whole bunch of modules, all in the same git repo, referencing their neighbours, so for example in our terraform_modules repo you have /cluster/{a,b,c}.tf and /ec2_assume_role_policy/module.tf, and /cluster/a.tf references module { source = "../ec2_assume_role_policy" } (and a whole lot other modules at the root of the repo).

Then, in our infra repo we have prod/infra.tf and dev/infra.tf, each files with essentially one module { source = "git://github.com/our_org/terraform_modules.git//cluster" }.

So, sorry if it was not intended for this but it was not clear at all that it was not a perfectly valid use case, and now we rely heavily on it :/…

[cep21]

For us, this is also a pretty serious regression. We have a single repository with many different terraform modules ,and some of them depend upon other modules in the same repository: referenced with source = '../name' I'm unsure what the alternative is for us, if this is permanent.

[nap]

We have a mono-repository for our project and we have both development and production folders in there. We have a terraform.tf in each that points to all the modules that we use.

For development, the terraform.tf points to the modules folder using relative path (../modules/MyModule). This way, we can test the latest version when we do terraform apply from the development folder.

In our production folder, in our terraform.tf, we use git tags to specify the source of the module. The fact that the whole source tree was checked out at the same time was a serious benefit for us. We sure that any submodules used by a tagged module was from the same git tag. We didn't have to also tag all the submodules and re-tagging them individually when we have to change development or production, which is error prone.

This enabled us to only update the tags our production/terraform.tf without touching anything else.

[dtserekhman-starz]

Hitting the same problem with relative paths for the module source specification. Please fix it. Thanks.

[navaati]

Thanks sir !

[nap]

Thank you.

[dtserekhman-starz]

Thanks!

[joestump]

Are there nightly builds or something? I'm blocked on this and can't downgrade since state has been applied using 0.10.6 already.

[ringods]

@joestump what we usually do as a workaround is patch the version number in the state back to 0.10.5.

[ethanfrogers]

Can anyone confirm that this issue is fixed? We have modules 2 levels up (../../moduleName/) and it is still broken in 0.10.7.

[ccmccull72]

This is still a problem, had to roll back version. I spent some time trying to figure out why suddenly didn't work, assuming it was my fault...

Specifically, we use init with a module directory to keep our plans, vars, and .terraform files out of the source. Please fix.

[nap]

If you refer to my usecase, it now works as intended.

[jbardin]

@ethanfrogers, @ccmccull72,

This issue is specifically about using a relative path (../) to locate a module "above" the path rooted at the named directory after //. If that's not working for you, please provide a an example that we can use to reproduce the problem. If you're having a different issue with the module directory layout, feel free to open a new issue with the problematic layout.

Thanks!

[cleverlight]

I think I've found an edge case of the same bug.

Terraform Version

$ terraform -v
Terraform v0.10.7

Terraform Configuration Files

./main.tf:

module "foo" {
  source = "./modules/primitive/foo"
}
output "out" {
  value = "${module.foo.out}"
}

./modules/primitive/foo/main.tf

module "bartab" {
  source = "../../bar"
}
output "out" {
  value = "${module.bartab.out}"
}

./modules/bar/main.tf

output "out" {
  value = "bar"
}

example.tar.gz

Terraform apply

A basic terraform apply works correctly:

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

out = bar

But based on the description of the apply command, I think this should work too:
terraform apply modules/primitive/foo

Expected result

As above

Actual result

Failed to load root config module: Error loading modules: module bartab: not found, may need to be downloaded using 'terraform get'

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.