Builtin support for templates

[errordeveloper]

When supplying user_data to a host, it's desirable to have input that can be planned/tested ahead of resource creation.

It specifically makes sense with static environment files and Cloud Config.

[errordeveloper]

I have currently approached this with a combination of file and exec provisioners, where I have a shell script that generates and environment file on the host based on the cloud provider and ${count.index} of the instance... But I think it would be a lot nicer to have ${template("cloud-config")}, which would potentially allow me to test it. Otherwise, #641 could help, that way I could at least lookup which file to pass to user_data.

I think that simple template engine with logic and scope bindings would be a great generic feature for Terraform. I suppose Go's text/template/ would be sufficient...

[errordeveloper]

Ok, no I looked back and can see that #237 was somewhat around a similar dilemma... So if we cannot bind all variables in the current scope, perhaps the user would have to chose what to bind to template processor, which would be reasonable. Although, one can now say, that if there is no fully automatic binding, what's the point of making template processor builtin? That may be a valid argument, but that would require one to satisfy dependencies on each host that may run it and inclusion of templates provided by 3rd-party would be uneasy because of that.

[armon]

Yeah I think the issue is we make heavy use of the declarative nature of TF. Using a variable is an implicit dependency that causes the graph traversal to be changed. This means you must declare any variable dependencies. This makes supporting templates like this very challenging. I think that there may be interesting ways to do this with a "logical" provider. (#580). Something like a "logical_template" which is able to declare dependencies and render a template as a resource. This resource would sit in the graph and restrict traversal orders, but then could be referenced as a variable. Anyways, food for thought.

[mitchellh]

The way we've wanted to add this, to add to what @armon said, is explicit templating.

Example feature that doesn't exist yet:

${template("file.txt", "foo=${var.foo}", "bar=${type.name.field}")}

Then you can use a templating language inside file.txt to access foo and bar.

[errordeveloper]

👍

[justinclayton]

👍

[justinclayton]

I should also mention that my interim solution for this was to break out make and sed. For anyone interested, the Makefile ended up looking something like this:

apply: plan
    terraform apply -input=false < plan

plan: cloud-config.yml
    terraform plan -input=false -out plan

cloud-config.yml: discovery_url
    cat templates/cloud-config.yml.template | sed -e 's#{{ discovery_url }}#`cat discovery_url`#' > cloud-config.yml

discovery_url:
    curl -s https://discovery.etcd.io/new > discovery_url

clean:
    rm -f plan discovery_url cloud-config.yml

Don't knock it, it works and is fairly readable!

[korya]

For everyone who uses CoreOS and wants to parametrize cloud-config.yml.

I've found a genious solution at https://github.com/emmanuel/coreos-skydns-cloudformation:

• use YAML anchors and aliases to parametrize cloud-config
• coreos-cloudinit uses go-yaml supporting the feature

So in your terraform config define your anchors, it should look like this:

resource "aws_launch_configuration" "service" {
  ...
  user_data = <<USER_DATA
#cloud-config

dynamic:
  fleet_metadata: &FLEET_METADATA
    metadata: instance_type=${var.aws_instance_type},public_ip=$public_ipv4,region=${var.aws_region},role=service
  discovery_url: &ETCD_DISCOVERY_URL
    discovery: ${var.etcd_discovery_url}
${file("cloud-config.yaml")}
USER_DATA
}

Then in your base cloud-config.yml use this anchors in the appropriate place, it should look like this:

coreos:
  fleet:
    <<: *FLEET_METADATA
    public-ip: $private_ipv4
  etcd:
    <<: *ETCD_DISCOVERY_URL
    addr: $private_ipv4:4001
    peer-addr: $private_ipv4:7001
  units:
    ...

The credit goes to @emmanuel.

[emmanuel]

Thanks for the props @korya; @payneio & I worked that out from past experience with hacking YAML-based configs 😄. This is working great for us (as of coreos/coreos-cloudinit@v0.10.0).

What I want now is a way to include instance-level metadata in fleet (e.g., EC2 instance IDs). But that's firmly a CoreOS/Fleet issue and not a Terraform one.

[josharian]

Would #1778 help with this?

[errordeveloper]

👍

[errordeveloper]

As #1778 provides the basics, I suppose I'd rather close this as it's pretty vaguely defined.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.