Inability to dynamically create resources via conditionals

[walterdolce]

Hi,

I was trying to leverage conditionals to create resources dynamically but it seems it's not actually possible. Please advise on whether I am missing something here or I'm doing something wrong.

# variables.tf
variable "environment" {
  default = "<<inherited_from_tfvars>>"
  type = "string"
}

variable "vpc_main" {
  default = {
    cidr_block = "<<inherited_from_tfvars>>"
    name = "main-vpc"
  }
  type = "map"
}

variable "vgw_propagated_list" {
  default = "<<inherited_from_tfvars>>"
  type = "string"
}


# resources.tf
resource "aws_vpc" "vpc" {
    cidr_block = "${var.vpc_main["cidr_block"]}"

    tags {
      Name = "${var.environment}-${var.vpc_main["name"]}"
      Environment = "${var.environment}"
    }
}

resource "aws_vpn_gateway" "vpn_gateway" {
  count = "${var.environment == "staging" || var.environment == "production" ? 0 : 1}"
  vpc_id = "${aws_vpc.vpc.id}"

  tags {
    Environment = "${var.environment}"
    Name = "${var.environment}-main-vgw"
  }
}

resource "aws_internet_gateway" "igw" {
  vpc_id = "${aws_vpc.vpc.id}"

  tags {
    Name = "${var.environment}-${var.igw_main_name}"
    Environment = "${var.environment}"
  }
}

resource "aws_route_table" "rt_public_1a" {
  propagating_vgws = ["${var.environment == "staging" || var.environment == "production" ? var.vgw_propagated_list : aws_vpn_gateway.vpn_gateway.id}"]
  vpc_id = "${aws_vpc.vpc.id}"

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = "${aws_internet_gateway.igw.id}"
  }

  tags {
    Environment = "${var.environment}"
    Name = "${var.environment}-${var.vpc_main["name"]}-rt-public-1a"
  }
}

Assume there is one tfvars per environment. When the var.environment is set to development, everything works fine as expected. The Virtual Private Gateway is created as expected (and successfully destroyed when not needed anymore).

But when the var.environment is set to staging or production, running terraform plan will produce:

Error running plan: 1 error(s) occurred:

* Resource 'aws_vpn_gateway.vpn_gateway' not found for variable 'aws_vpn_gateway.vpn_gateway.id'

Looks like Terraform is expecting the VPN Gateway resource to exist when in reality it shouldn't.

To rephrase: the VPN Gateway resource will have count = 0 when the var.environment is set to staging or production, and it should pick the var.vgw_propagated_list variable instead. By doing so, I am effectively declaring a dynamic resource which would be created based on the environment you're working on. But it's not working.

Is this expected? Am I doing something wrong?

Thank you all.

[mtougeron]

This would be extremely useful for me as well. I was trying to add a toggle between using an aws ELB or ALB (using count like the example above) and ran into this issue for the route53 record

resource "aws_route53_record" "alb" {
  name = "${var.application-name}-${var.environment}"
  type = "A"

  zone_id = "${module.aws.route53-zone-id}"

  alias {
    name = "${var.use_elb == "1" ? aws_elb.default.dns_name : aws_alb.default.dns_name}"
    zone_id = "${var.use_elb == "1" ? aws_elb.default.zone_id : aws_alb.default.zone_id}"
    evaluate_target_health = false
  }
}

EDIT: actually, a better example is with the aws_autoscaling_group resource

resource "aws_autoscaling_group" "default" {
  name = "${var.application-name}-${var.environment}"

  target_group_arns = ["${var.use_elb == "0" ? "${aws_alb_target_group.default.arn}" : ""}"]
  load_balancers = ["${var.use_elb == "1" ? "${aws_elb.default.name}" : ""}"]

[apparentlymart]

Right... the problem here is that the interpolation language isn't currently able to "short-circuit" the evaluation of the conditional branch that doesn't end up being used. This is a known limitation of the language evaluator currently, and one I'd definitely like to fix. Thanks for capturing this in an issue!

[walterdolce]

Thank you for taking it into consideration. I look forward to having this capability!

[apparentlymart]

I've filed hashicorp/hil#50 to track the root issue here; the "hil" codebase is where the interpolation language infrastructure lives.

[alexsomesan]

I found a usable workaround to this issue. It unblocked us, so maybe it can work for others too.
You can see it being used here: https://github.com/coreos/tectonic-installer/blob/master/modules/aws/vpc/vpc.tf

It's based on the fact that resources which have a count attribute can be applied splat syntax .*. to obtain lists / arrays of attributes. Lists can be empty, so they won't cause an error during ternary operator evaluation. So if you need to condition a resource creation with a count of 0 or 1, you can then refer the count-gated resource like a list of resources when you need an attribute of it.
You can then get a "scalar" value of it by applying a join() to the resulting list of attribute, since the list has at most one element. If you do need to enable higher count instances, just wrap a split() around the join.

I know this is very ugly and downright abusive to HCL, but it does work around the problem. Ideally a fix upstream in HIL will render this hack useless.

[jennyfountain]

Seeing the same issue here too. Help! :D

[apparentlymart]

The workaround of using the "splat" syntax is the best path for the immediate term, though I totally understand that it is ugly and counter-intuitive. Others have reported success with it, and it's become a common idiom while we work through this problem.

On a more positive note, we're working right now on a revamp of the configuration language that includes, among many other fixes and improvements, a proper solution to this problem. Given the scope of the changes we will be rolling this out carefully, first via an opt-in experimental version to gather feedback on some of the more significant changes.

The new work here changes a few different things that related to this issue:

• The ... ? ... : ... conditional operator now does correctly ignore errors coming from the unselected branch of the conditional, solving the direct problem here and allowing the more intuitive usage.
• The behavior of count and the "splat" construct is altered slightly to remove some of the strange behaviors it currently has. In particular, the * is now a general operator that can be applied to any list, and as a result a counted resource can appear as a true list in the language, making it compatible with other list-expecting operations.

We'll have more info on this once it gets closer to being released. As noted above, the first release will be an opt-in, experimental version so we can gather feedback and address any big issues before we switch to the new implementation. However, we are definitely motivated to address these issues as quickly as possible since we know that these assorted inconsistencies all add up to making Terraform feel clumsy to use and make Terraform configurations hard to maintain as a result.

[thallam08]

I've also had this issue, with an additional twist: If the reference to the missing resource is part of a conditional that sets the value of a map item, then there is a silent failure but interpolation of other items in the map fails.

ie in the example below, used as output, "fixedItemID" is set to an empty string rather than the interpolated value

creationFlag = false

output myMap  {
  conditionalItemID = "${ var.creationFlag ? aws_instance.MissingItem.id : aws_instance.CreatedItem.id }"
 fixedItemID = "${aws_instanceItem.fixedItem.id}"
}

[FrenchBen]

Bump to this issue; as seen with the many related Issues/PR, it's amazing that a year later the latest interpolation parser hasn't been pushed.
Ran into this issue today, while trying to do something pretty simple:

variable "file_path" {
	default = ""
}

locals {
  content = "${var.file_path != "" ? file(var.file_path) : "initial content here"}"
}

It's pretty annoying that this doesn't work as expected and that both side of the ternary operation are actually interpreted.

ps: for those interested, coalesce is a good workaround - Using the above, it would be transformed to:

locals {
  content = "${var.file_path != "" ? file(coalesce(var.file_path,"/dev/null")) : "initial content here"}"
}

similar to the solution found here: #15605

[jurajseffer]

@apparentlymart I think I'm hitting this or a very similar problem of Terraform evaluating the entire interpolation rather than just the "true" part in this issue hashicorp/terraform-provider-aws#3744

[MMarulla]

It's been almost a year and a half since a fix to the configuration language was mentioned. This item is still open. Update???

[paddie]

The ternary operator issue it's fixed in HCL2, there is a beta version available but it's not our of Beta I believe..

…

On Thu, Mar 7, 2019, 18:51 MMarulla ***@***.***> wrote: It's been almost a year and a half since a fix to the configuration language was mentioned. This item is still open. Update??? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11566 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAaEQq0noS4SNpEfTQCm_br6KK0HUVmCks5vUVGKgaJpZM4Ly9Ag> .

[JustinGrote]

@paddie is it part of the latest RC1?

[mildwonkey]

@JustinGrote (and everyone else): yes, short-circuit conditional evaluation is in terraform 0.12. You can test it now in terraform 0.12.0-rc1 - but please do not use a pre-release with real production infrastructure!

[JustinGrote]

Confirmed, this plus the "null" value opens up a LOT more scenarios to make modules more friendly with intelligent defaults and "plug-in" objects. Already looking forward to hopefully for_each and count for modules in 0.13+!

…

On Tue, May 21, 2019 at 11:43 AM Kristin Laemmert ***@***.***> wrote: @JustinGrote <https://github.com/JustinGrote> (and everyone else): yes, short-circuit conditional evaluation is in terraform 0.12. You can test it now in terraform 0.12.0-rc1 <https://releases.hashicorp.com/terraform/0.12.0-rc1/> - but please do not use a pre-release with real production infrastructure! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11566?email_source=notifications&email_token=ADUNKUUOVOV5AJSGNIHOP4TPWQ7DHA5CNFSM4C6L2AQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV4Z7JQ#issuecomment-494509990>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADUNKUUDFBEMJ5OM4NM52ALPWQ7DHANCNFSM4C6L2AQA> .

[mildwonkey]

🎉 I am going to close this issue; the original bug is fixed and working as expected in Terraform 0.12. Thanks for the great conversation!

[thallam08]

The use of count like this is really a hack, especially as true / false are no longer cast to 1 and 0.

Ideally there needs to be a feature request, for a true conditional resource (ie resource only created if resource.CreateResource is 'true' )

Currently I have to set count to the result of a conditional statement that has the value 0 if false, and the number of items configured in the environment if true. I'm controlling features of the environment independently to the number of items that would be created if the feature was enabled.

[mildwonkey]

Hi @thallam08! Please open a new GH issue and use the feature request template to request an enhancement - the more specific your example use-case, the better. Thanks 🎉

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.