depends_on cannot be used in a module

[chrisrlong]

Hi there,

Terraform Version

0.8.0 rc1+

Affected Resource(s)

module

Terraform Configuration Files

module "legacy_site" {
  source = "../../../../../modules/site"
  name = "foo-site"
  health_check_target = "TCP:443"
  azs = "${var.azs}"
  instance_count = "${var.instance_count}"
  vpc = "apps"
  region = "${var.region}"
  environment = "${var.environment}"
  run_list = "hs_site_foo"

  #rds_complete = "${module.rds.db_instance_id}"
  #elasticache_cache_complete = "${module.elasticache_cache.elasticache_id}"
  #elasticache_sessions_complete = "${module.elasticache_sessions.elasticache_id}"

  depends_on = [
  "module.rds",
  "module.elasticache_sessions"
  ]

}

Debug Output

Error loading Terraform: module root: module legacy_site: depends_on is not a valid parameter
module root: module legacy_site: depends_on is not a valid parameter

Expected Behavior

I am trying to use the new depends_on instead of the above outputs, so I create and provision my app once I know database and caches are built.

Actual Behavior

Nothing as terraform errors out as above.

Steps to Reproduce

1. terraform apply

References

depends_on can reference modules. This allows a resource or output to depend on everything within a module. (#10076)

[stack72]

Hi @chrisrlong

This is very strange - we introduced depends_on for modules in 0.8.0-beta2

can you run terraform version and post the output here for me?

Paul

[chrisrlong]

Hi @stack72,

I have complied master from version b15b8bd (Terraform v0.8.0-dev (b15b8bd+CHANGES)), which should include beta2 features. (I did this to get 10337 and 10338 fixes)

It seems it would work if you set a resource to depend on a module, but you cannot set a module to depend_on another module......

Thanks for the great tool btw ;)

Chris

[stack72]

Hi @chrisrlong

thanks for getting back to me - you are indeed correct - modules cannot (currently) depend_on other modules but can depend on a resource only

Will change the tag on this from bug to enhancement :)

Keep an eye out for it soon ™️

Thanks

Paul

[mitchellh]

Retagging as enhancement, since this isn't broken functionality, its functionality that doesn't exist yet. :)

[sheerun]

My use case: the slave virtual machine depends on master virtual machine to exist

[shantanugadgil]

I am seeing the same error:
module root: module example: depends_on is not a valid parameter

My Terraform version is:
Terraform v0.8.2

My use case is that I want a module to depend on a resource.

[arehmandev]

Error loading Terraform: module root: module etcdbastion: depends_on is not a valid parameter.
^ Seeing it here too

[cemo]

@mitchellh I believe this one can solve unnested modules problem too. There were some issues regarding this #10883 before. This solution would be intuitive and address these issues.

[vladimir-kozyrev]

Waiting for this feature to be added 👍

[AirbornePorcine]

Anyone have a good workaround for this in the meantime?

[jacobwgillespie]

My pseudo workaround is to add a list variable inside the module:

variable "depends_on" { default = [], type = "list" }

Then when using the module, pass it a computed value from the resource I want it to depend on:

module "something" {
  depends_on = ["${aws_instance.instance.private_ip}"]
}

[AirbornePorcine]

Thanks, that doesn't entirely fit what I think this issue is talking about though - making a module able to depend on another one. I don't have any resources that the module should depend on - it should depend on another module being created.

I've tried doing stuff like this:
module 1:

output "wait_for_cluster" { value = "ref to a resource that gets created last by this module" }

module 2:

variable "wait_for_cluster" {}

Main terraform template:

module "1" {

}

module "2" {
 wait_for_cluster = "${module.1.wait_for_cluster}"
}

But this doesn't do anything - my module two is still created at basically the same time as module 1.

[b-dean]

@AirbornePorcine In your second module you need to actually use the wait_for_cluster variable somewhere that establishes a dependency. Such as in a template or a trigger on a null_resource, etc.

See #1178 (comment) from @kristjanelias

The only change I'd make to his workaround is to maybe make the dummy dependency resource use a trigger so it changes when the instance changes.

resource "null_resource" "dummy_dependency" {
  triggers {
    dependency_id = "${aws_instance.instance.id}"
  }
}

[AirbornePorcine]

Ah, that makes sense! Should have known that Terraform would optimize away my variable if I didn't use it. :) I tried out your suggestion with the trigger and that didn't do it, but setting the depends_id on a tag on some resource in module 2 seems to have done it. Hacky but oh well I guess.

[jspiro]

Thank you @danieldreier – epic update and the transparency is appreciated! I think we can finally all rest easy that this is coming ^_^

[chrisrlong]

Thanks is good news @danieldreier. Time will come to remove all the work arounds soon then...

[jared2501]

Glad to hear! super exciting

[ortelli-oxalide]

Thanks @danieldreier we are looking forward to that !

As I cannot see anyone else reporting a specific issue related to module dependencies that I'm encountering, I'll report it here. I hope that the support of "depends_on" on modules will resolve that, but I'm not quite sure. Any idea ?

With some code like this:

module "some_module" {
  ...
  some_parameter = module.another_module.output
  ...
}

# some_module source code
resource "some_resource" "_" {
  count = length(var.send_rds_logs_to_dd_with_lambda_arn)
  ...
}

Terraform raise the following error (at plan or apply):

Error: Invalid count argument
...
in resource "some_resource" "_":
   2:   count = length(var.send_rds_logs_to_dd_with_lambda_arn)

The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.

[matti]

I think I managed to create a valid workaround for this: https://github.com/matti/terraform-module-depends_on

It requires modules to have some boilerplate, but otherwise it seems to work fine.

[kuwas]

@danieldreier @apparentlymart Thanks for the update and putting the emphasis on this, it is greatly appreciated.

[solidDoWant]

@danieldreier Great to hear you guys are working on this. Have you guys been able to scope and figure out a timeline for the project yet? Roughly when this is planned to be released is one of the deciding factors for if my team will be using TFC or switching to Pulumi. Can you provide any broad information on when this will be released (week/month/year)?

Edit: Others might still need an ETA but we don't. The lack of communication (both on GitHub and via support email) about any estimates as to when Hashicorp will complete features they've committed to has resulted in my team canceling our TFC subscription. We're now moving to Pulumi.

[danieldreier]

@solidDoWant While I appreciate the need for you and others to plan, we are not yet in a position to communicate any timelines. My hope above was to begin communicating what we’re currently focused on more publicly.

I can give a few details on the larger first-class module project that this is a part of to help you with your planning:

• improving modules is the main feature of the next terraform release: we're expecting to ship make modules first-class in 0.13, which is the next major release. The only other thing that we'll block 0.13.0 on is a set of critical bug fixes that require backward-incompatible behavior changes. We're not planning on holding back 0.13.0 for any other features.
• depends_on may ship after count and for_each: depends_on is a subset of the larger project of first-class modules. As part of that project, we'll make depends_on, count, and for_each work in modules. I can't tell you, yet, whether we'll ship depends_on, and so count and for_each all at once, or what order they'll ship in - we may ship for_each and count in 0.13.0 and then quickly follow up with a minor release that adds depends_on. We do intend to batch any breaking changes in 0.13.0, so that the later addition of depends_on would be purely additive.

[jspiro]

That prioritization makes a lot of sense to me. Looking forward to it!

[pixelicous]

Great, 3 years in, looking forward to having this needed feature.

[hakro]

This is a really important feature to have in my opinion. It would make the code much more readable and maintainable than by using one of the hack that we find on blog posts.

[emmaLP]

Can anyone please give a time line on when this feature will be released?

[max-l-weaver]

Wow, over three years and still nothing?!

[matti]

three years is nothing in terraform time

[ketzacoatl]

Wow, over three years and still nothing?!

@max-l-weaver with this comment, you are showing how unappreciative and ignorant you can be.

You can fault the devs for not getting this right with modules from the start, but aside from that, you should have more awareness of reality and what these changes have required before sharing those types of crass comments.

In fact, you should probably not use any open source software, as the type of comment you left above are one of the major reasons why application developers give up on open source.

Please be more kind and appreciative in your comments, or leave them to yourself.

[solidDoWant]

Just because the software is open source doesn't mean we aren't paying for it. Many of us (myself previously included) pay a monthly or yearly fee for Terraform/TFC and in term expect support and timely software updates. The fact that Hashicorp rarely if ever shares roadmaps or any kind of even a guess at a timeline quite frankly makes it unusable for business critical services, such as the infrastructure backing software company's services. The lack of communication on one of, if not the most requested feature of the last 3-4 years is not something to appreciate.

[confiq]

@solidDoWant , chillrex man... Many people "pay a monthly" and wait patiently for a new feature.
This specific feature requires lot of tech resources and time, no need for such an outcry because it does not help anyone.

They decided to implement it which is great news, crass comments like these do not help anyone.

[kuwas]

@solidDoWant There are various workarounds for this feature that currently exists, such using templates to generate tf files, so its not really a showstopper. We should be thankful that they even prioritized this at all.

[solidDoWant]

To clarify, my issue isn't at all with the lack of the feature - it's with the lack of communication. I would be fine with Hashicorp never developing this feature, but I take issue with Hashicorp not being able to even take a guess as to when it will be completed, if at all. It's great that they've communicated that this is a top priority, as that address the 'if it will happen', but they haven't addressed the 'when'.

This Github issue isn't the only place affected by the underlying company issue either. There are 136 PRs on this repo alone (not to mention the numerous providers) that are open. Out of those 135, only one has been assigned a milestone. Roughly 15 haven't even been commented on. On most of these there is no indication when, or if these PRs will be addressed. Not great from a user's perspective when there's a fix you're waiting on in one of them, or in a developer's perspective when your PR sits open for months or years.

Hashicorp makes some cool tools and (from what I've seen) most of their code is pretty solid. I've no issues with the software developers, but their product strategy and overall communication leaves a lot to be desired.

[camjackson]

@danieldreier can you please lock this issue to contributors only? I believe at this point just about every constructive comment that could be made, has already been made.

There are likely hundreds of people subscribed to this issue, and we don't all need to get an email every time someone wants to complain. I'll be happy if the next time I hear about this feature is when it's done.

[ketzacoatl]

To clarify, my issue isn't at all with the lack of the feature - it's with the lack of communication.

Thanks for clarifying.

I would be fine with Hashicorp never developing this feature, but I take issue with Hashicorp not being able to even take a guess as to when it will be completed, if at all. It's great that they've communicated that this is a top priority, as that address the 'if it will happen', but they haven't addressed the 'when'.

I can understand that frustration, I agree Hashicorp could improve their communication with us as users of various kinds.

However, WRT this general topic of first-class modules, I believe Hashicorp has been pretty clear, and within the bounds of what's possible (and also avoiding making unnecessary expectations which they may fail to deliver on).

For example, in the 3 years that this has transpired, we've seen the following:

1. many, rather verbose and elaborate discussions of the use cases and needs and possibilities, the problems and blocks, as well as how to get there.
2. Hashicorp realizes their fsck up, and acknowledges that TF core needs to be refactored in major ways.
3. Hashicorp lays down the roadmaps for 0.11 and 0.12, with various breaking/major changes that will facilitate the next round of changes
4. Hashicorp plows through 0.11 and 0.12 releases, following through as planned.
5. Hashicorp starts work on 0.13 which is focusing on modules as a first-class citizen

Particularly for those last two, no timelines (with unreasonable expectations) were laid down. Yea, I would have loved to see 0.12 completed sooner, or dev on 0.13 to have started sooner, but I'd rather this than to have a product which is all over the place, poorly developed, poorly designed, lots of major breaking changes, and just generally moving to the whims of the wind as it blows.

[max-l-weaver]

@ketzacoatl please take a deep breath and calm down.

Imo Hashicorp market themselves as a software provider that provides solutions for businesses. If it was some dude working on his own in his basement then of course i wouldn't post such ignorant comments, however the fact that this feature has been open for over three years with no communication is a little disconcerting. I apologise if you felt somehow attacked by my comment.

[phildier]

Please take the complaints to twitter or reddit. I'm subscribed to this thread for news about the feature only.

[danieldreier]

Based on the feedback here, I've locked this discussion. We're actively working on module expansion. The engineers are making good progress, and I am excited about 0.13. For folks who are really upset, I'm ddreier@hashicorp.com and I'm happy to talk.

[danieldreier]

I'm very excited to announce that beta 1 of terraform 0.13.0 will be available on June 3rd, and will include module depend_on. I've pinned an issue with more details about the beta program, and posted a discuss thread for folks who want to talk about it more.