Cannot pass output string of one module as input list to another module #12263

ordinaryworld · 2017-02-26T15:59:45Z

Hi,

I have two modules:

module 1 loadbalancer:

...
output "backendAddressPoolId" {
  value = "${azurerm_lb_backend_address_pool.backendAddressPool.id}"
}

module 2 virtual machine:

...
variable "lbBackendAddressPoolIds" {
  type = "list"
  default = []
}

resource "azurerm_network_interface" "nic" {

  count               = "${var.count}"
  name                = "${var.prefix}-nic-${var.name}-${count.index+1}"
  location            = "${var.location}"
  resource_group_name = "${var.resourceGroupName}"

  ip_configuration {
    name                          = "ipConfigNode"
    subnet_id                     = "${var.subnetId}"
    private_ip_address_allocation = "Dynamic"
    load_balancer_backend_address_pools_ids = "${var.lbBackendAddressPoolIds}"
  }

}

If I pass the output of the first module to the second module as shown below:

module "loadBalancer" {
  source = "module1"
}

module "vm" {
  source = "module2"
  lbBackendAddressPoolIds = ["${module.loadBalancer.backendAddressPoolId}"]
}

Then I get the following error on terraform plan command

$ terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * azurerm_network_interface.nic: ip_configuration.0.load_balancer_backend_address_pools_ids: should be a list

I noticed that if I hard code the value in output then it all works without any errors!

module 1 loadbalancer with hardcoded output value:

...
output "backendAddressPoolId" {
  value = "test123"
}

Note sure why using a computed value in the list is failing. Any help will be appreciated!

Terraform Version: v0.8.7

The text was updated successfully, but these errors were encountered:

apparentlymart · 2017-02-26T17:20:41Z

Hi @ordinaryworld! This is some confusing behavior, I agree...

My guess (not actually tested) is that the computed-ness of the list is causing Terraform to see it as a string due to some details of how Terraform currently represents computed values internally.

You might be able to trick Terraform into doing the right thing here by presenting the variable as an interpolation into a list, like this:

load_balancer_backend_address_pools_ids = ["${var.lbBackendAddressPoolIds}"]

ordinaryworld · 2017-02-27T04:12:47Z

@apparentlymart Thanks! That worked! Just wondering if this is this documented somewhere?

jbardin · 2017-03-14T20:29:26Z

@ordinaryworld,

This is in the Interpolation documentation, under the "User list variables" heading.

I'm not entirely certain what we can do to make this easier to spot, the error is precise.
@apparentlymart, do you happen to know of there's enough context at the point of the error to be able to suggest wrapping the interpolation in [ ]?

mgresko · 2017-09-08T17:39:50Z

I see something similar in that I can't take a module output that is a list and use it as a list input for another module. No amount of [ ] fixes.

$ terraform --version
Terraform v0.10.4

* Terraform's "type system" sometimes doesn't identify list types correctly so be explicit * hashicorp/terraform#12263 (comment)

horsey · 2018-01-26T11:13:25Z

I am facing the same thing when using Terraform v0.11.1.
I can get around the problem by adding [] around the variable in the module that receives the list input as a string, however, I have trouble on how to manage this when the source does not define the variable explicitly and sends an empty array instead. Any suggestions or workarounds would be appreciated.

kmcquade · 2018-03-08T20:25:38Z

Agreed. This is a total pain. Would love to be able to pass lists through to a module

mlvnds · 2018-03-12T14:10:59Z

👍

Same here with:

module_main.tf

resource "aws_mq_broker" "default" {
  broker_name         = "${var.amazonmq_broker_name}"
  engine_type         = "${var.amazonmq_engine_type}"
  engine_version      = "${var.amazonmq_engine_version}"
  host_instance_type  = "${var.amazonmq_instance_type}"
  publicly_accessible = "${var.amazonmq_publicly_accessible}"
  security_groups     = "${var.amazonmq_security_groups}"

  user {
    console_access = "${var.amazonmq_user_console_access}"
    groups         = "${var.amazonmq_user_groups}"
    password       = "${var.amazonmq_user_password}"
    username       = "${var.amazonmq_user_username}"
  }
}

module_variables.tf

[...]
variable "amazonmq_security_groups" {
  type = "list"
}

environment_main.tf

[...]
module "amazonmq" {
  source = "../../modules/amazonmq"

  amazonmq_broker_name     = "${var.amazonmq_broker_name}"
  amazonmq_security_groups = "${list(module.vpc.vpc_sg_default)}"
  amazonmq_user_password   = "${var.amazonmq_user_password}"
  amazonmq_user_username   = "${var.amazonmq_user_username}"
}

Where ${module.vpc.vpc_sg_default} is a string gathered from:

[...]
output "vpc_sg_default" {
  value = "${aws_default_security_group.default.id}"
}

Meanwhile...

As @horsey has mentioned above, the bracket trick works fine when list is not empty.

module_main.tf

resource "aws_mq_broker" "default" {
  [...]
  security_groups     = ["${var.amazonmq_security_groups}"]
  [...]

VipinAgarwal · 2018-07-23T23:37:16Z

Not sure why we need to add [.....], but worked like a charm. :)
Workaround to send the empty list issue - I used compact and it will reduces the list from [""} to []

source_application_security_group_ids =["${compact(split(",", "${lookup(var.security_rules_list[count.index], "source_application_security_group_ids", "")}"))}"]

royerhq · 2018-09-19T20:03:26Z

I had the same issue, adding the [ ] to the resource solve the issue.

So from this

  name  = "${upper(var.aws_region_code)}-${upper(var.aws_project_code)}-ALB-${var.alb_number}"
  load_balancer_type  = "application"
  internal            = "${var.lb_internal}"

  security_groups     = "${var.lb_security_groups}"   <----- Issue Here
  [...]
}

To this

  [...]
  security_groups     = ["${var.lb_security_groups}"]   <----- Fixed Here
  [...]
}

apparentlymart · 2018-10-26T21:59:52Z

Hi all!

The Terraform v0.12.0-alpha1 release includes the initial fix for this problem, allowing a list with unknown elements to pass through module boundaries while retaining its type as a list. This means the strange workaround of adding an extra level of list brackets is no longer needed, and in fact must no longer be used because it leads to Terraform understanding the value as a list of lists, rather than a single list.

As noted in #19140, as of the alpha1 release the error message for having the redundant brackets is not particularly helpful. We're hoping to introduce a more specific, actionable error message before v0.12.0 final, but either way this is also a change that we intend to fix automatically as part of the configuration migration tool that will be included in v0.12.0 final, thus making the fix easy once Terraform has detected and reported the problem.

Since the original problem reported here is now fixed in the master branch, and we have #19140 covering the remaining usability problem, I'm going to close this out. Thanks for reporting this, and sorry for the delay in getting it fixed.

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` It is less than ideal to do all this user logic in the module, but MVP and HCL 2 will fix all the worlds problems. [1] hashicorp/terraform#12263

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` [1] hashicorp/terraform#12263

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` It is less than ideal to do all this user logic in the module, but MVP and HCL 2 will fix all the worlds problems. [1] hashicorp/terraform#12263

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` [1] hashicorp/terraform#12263

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` It is less than ideal to do all this user logic in the module, but MVP and HCL 2 will fix all the worlds problems. [1] hashicorp/terraform#12263

* Initial module implementation Note that the template is the default XML provided by AmazonMQ * Change output format due to Terraform issue Currently a plan is failing with lots of: ``` * module.amq.output.secondary_ampq_ssl_endpoint: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.endpoints.1' for variable 'aws_mq_broker.default.*.instances.1.endpoints.1' * module.amq.output.secondary_ip_address: Resource 'aws_mq_broker.default' does not have attribute 'instances.1.ip_address' for variable 'aws_mq_broker.default.*.instances.1.ip_address' * module.amq.output.primary_console_url: Resource 'aws_mq_broker.default' does not have attribute 'instances.0.console_url' for variable 'aws_mq_broker.default.*.instances.0.console_url' ``` [1] hashicorp/terraform#16681 * Move aws_mq_broker users into module These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` It is less than ideal to do all this user logic in the module, but MVP and HCL 2 will fix all the worlds problems. [1] hashicorp/terraform#12263 * Update docs * Move key_id into local So we don’t duplicate the join/splat pattern * Remove broker_name variable This should just be `module.label.id` for consistency * Consistent ordering of input variable params

These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` [1] hashicorp/terraform#12263

* Add basic main.tf This will hold our global config to be used in other resources * Add default global variables * Add Postgres Aurora This commit has largely been lifted from our backing services module for Postgres Aurora. It creates a Postgres Aurora cluster and writes the outputs of that to SSM. If no master username / password is given, these are generated as a random strings and saved to SSM using SecureString where necessary. If not postgres_name is given, it defaults to using using the terraform-null-label outputted id. * Add VPC backing-services This is part of terraform-root-modules backing service but I think we will likely want to create the VPC outside of this module and pass it in. I’ve left in for now but we can take out later. * Add Postgres read replica * Add Elasticache Redis backing service * Add Kops metadata module This is used to pull information from our kops cluser by doing a load of data lookups based on some tags. We can then pass things like Kops node security groups around to be used by other modules. * Add AmazonMQ backing service for CodeFresh This commit adds ActiveMQ broker and config for CodeFresh Enterprise. An admin user is created and credentials stored encrypted in SSM. A DNS hostname is created for the ActiveMQ endpoints. Note that unlike the other backing modules in here, AmazonMQ resources are not currently their own module. There are only a handful of resources for this AmazonMQ stuff but we can pull it out into a module if we so wish. The default ActiveMQ config [1] has been added. This is optional in the `aws_mq_broker` but due to Terraform not supporting conditional blocks beyond a basic count, it is a pain to conditionally add this. The schema can be found [2] [1] http://svn.apache.org/repos/asf/activemq/trunk/assembly/src/release/conf/activemq.xml [2] https://s3-us-west-2.amazonaws.com/amazon-mq-docs/XML/amazon-mq-active-mq-5.15.0.xsd * Add docs * Remove RDS Aurora Postgres replica * Remove VPC and subnet modules We will be injecting these resources into an existing VPC * Remove need for kops metadata * Move AmazonMQ into own module * Add EFS * Drop Redis Elasticache version to 3.2.6 3.2.10 doesn’t support encryption, see below: ``` Error creating Elasticache Replication Group: InvalidParameterCombination: Encryption features are not supported for engine version 3.2.10. Please use engine version 3.2.6 ``` * Move aws_mq_broker users into module These should have been in the module that calls this module however there is a Terraform bug [1] meaning passing the list of user maps is failing when trying to use a local value. From the calling module, the below does work: ``` users = [{ "username" = "admin" "password" = "defaultpassword" "groups" = ["admin"] "console_access" = true }] ``` however, using locals as we want to, it does not: ``` users = [{ "username" = “${local.admin_user}” "password" = “${local.admin_password}” "groups" = ["admin"] "console_access" = true }] ``` [1] hashicorp/terraform#12263 * Update docs * Remove deprecated mq_broker_name variable * Pin aws-mq-broker module to 0.1.0 release * Add global enabled variable for whole module We can toggle individual components of the backing services by enabling a specific service, or using the global flag to enable/disable all. * Add s3 bucket to CodeFresh backing services. * Rename node_security_groups to security_groups These maybe the security groups of your kops nodes, they may not, so be more generic. * Add usage to README * Pass only 1 or 2 subnets to mq.tf Running ActiveMQ in ACTIVE_STANDBY_MULTI_AZ mode requires you only supply 2 subnets. Any more and the resource will complain. Similarly you must pass a single subnet if running in SINGLE_INSTANCE mode * Actually use postgres_db_name if we pass it in * Add full example * Remove postgres_name variable This should just be var.name as the postgres name is computed in the RDS cluster module using labels. * Pin mq broker module to latest 0.2.0 release * Remove redis_name as this is calculated in module * Update Redis variable descriptions * overwrite SSM parameter is expected as a boolean * Bump AmazonMQ default instance type mq.t2.micro is likely not what you want in production. * Remove null-label since not being used anymore It is in use in all the submodule we call and we delegate down to them for naming. * Bump aws-efs module To ensure we can use enabled flags on it * Bump aws-s3-bucket to 0.1.0 Since we have the initial PR merged and have cut a release * Remove aws-mq-broker enabled flags Bump aws-mq-broker to 0.3.0, which does not feature enabled flags, so remove them. In its current incarnation the terraform-aws-mq-broker does not support the enabled variable allowing for boolean creation of resources in the module, see [1] for more context. [1] cloudposse/terraform-aws-mq-broker#4 * Add optional EFS VPC and subnet_id variables This commit adds variables to enable overriding what VPC and subnet that EFS runs in. If you don’t provide them, it default back to the `var.vpc_id` and `var.subnet_ids` values. During testing of this module, we found something interesting. We were deploying EFS to the backing services VPC, which is a different VPC to the k8s cluster. Our pods were unable to resolve the DNS endpoint of the EFS cluster, despite there being VPC peering between the two, with DNS lookups between them enabled. AWS documentation [1] states that in this scenario, you have a few options: 1) Use the EFS IP address directly (no thanks) 2) Create a DNS entry in Route53 CNAME’ing to the EFS DNS endpoint in a private hosted zone The underlying EFS module does already create a Route53 DNS entry CNAME to the EFS DNS endpoint, but it isn’t in a private zone. The pod could not resolve the Route53 DNS. Deploying EFS into the _same_ VPC as the k8s nodes worked a treat and finally the pods were able to resolve and mount EFS volumes. [1] https://docs.aws.amazon.com/efs/latest/ug/manage-fs-access-vpc-peering.html * Fix typos * Fix typos * Remove EFS + AmazonMQ from CodeFresh services CodeFresh Enterprise is not compatible with AmazonMQ on the protocol level and EFS will be moved into a general Kops backing service. * Remove Terraform glue variables These should be in the caller module, not this generic module. * Update docs and pin example modules * Update docs to remove TODO and add note on enabled

ghost · 2020-04-01T02:17:13Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

mitchellh added bug core labels Feb 27, 2017

kenanbiel mentioned this issue Sep 14, 2017

Multiple resources being spawned when passing output string from one module as input to another module #16094

Closed

dghubble added a commit to poseidon/terraform-render-bootstrap that referenced this issue Oct 3, 2017

Add square brackets for lists to be explicit

cc2b457

* Terraform's "type system" sometimes doesn't identify list types correctly so be explicit * hashicorp/terraform#12263 (comment)

apparentlymart added config and removed core labels Mar 10, 2018

apparentlymart closed this as completed Oct 26, 2018

apparentlymart added this to the v0.12.0 milestone Oct 29, 2018

phillbaker mentioned this issue Nov 5, 2018

Interpolation Support for URL phillbaker/terraform-provider-elasticsearch#11

Closed

dan-hill2802 mentioned this issue Nov 13, 2019

allow optionally enabling access logs on mail list & template bucket dwp/terraform-aws-ses-notification-service#4

Merged

ghost locked and limited conversation to collaborators Apr 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot pass output string of one module as input list to another module #12263

Cannot pass output string of one module as input list to another module #12263

ordinaryworld commented Feb 26, 2017

apparentlymart commented Feb 26, 2017

ordinaryworld commented Feb 27, 2017

jbardin commented Mar 14, 2017

mgresko commented Sep 8, 2017 •

edited

Loading

horsey commented Jan 26, 2018

kmcquade commented Mar 8, 2018

mlvnds commented Mar 12, 2018

VipinAgarwal commented Jul 23, 2018 •

edited

Loading

royerhq commented Sep 19, 2018 •

edited

Loading

apparentlymart commented Oct 26, 2018

ghost commented Apr 1, 2020

Cannot pass output string of one module as input list to another module #12263

Cannot pass output string of one module as input list to another module #12263

Comments

ordinaryworld commented Feb 26, 2017

apparentlymart commented Feb 26, 2017

ordinaryworld commented Feb 27, 2017

jbardin commented Mar 14, 2017

mgresko commented Sep 8, 2017 • edited Loading

horsey commented Jan 26, 2018

kmcquade commented Mar 8, 2018

mlvnds commented Mar 12, 2018

Same here with:

Meanwhile...

VipinAgarwal commented Jul 23, 2018 • edited Loading

royerhq commented Sep 19, 2018 • edited Loading

apparentlymart commented Oct 26, 2018

ghost commented Apr 1, 2020

mgresko commented Sep 8, 2017 •

edited

Loading

VipinAgarwal commented Jul 23, 2018 •

edited

Loading

royerhq commented Sep 19, 2018 •

edited

Loading