Official MIME Type for HCL?

[hairyhenderson]

(Apologies if this is in the wrong location, please redirect me if so!)

I'm looking to add HCL support to gomplate, and in order to do so I need a MIME type to identify the format with.

Unless I'm mistaken (I didn't do that much searching, after all), I don't think there's any conventional (or canonical) type already in use. Would the maintainers of this repo have any objection to application/hcl? Or would you prefer something else?

It doesn't need to be IANA-registered to be in use, but if you're interested in registering application/hcl (or something else), here is the place to start: https://www.iana.org/form/media-types.

[apparentlymart]

Hi @hairyhenderson,

We've not yet strongly considered a media type for HCL because thus far we've not used it in contexts where MIME types are relevant.

I went hunting for some precedent for what media types had been selected for other similar formats like YAML, TOML, UCL etc and was amused to find the very similar toml-lang/toml#465! Sadly, the precedent doesn't seem super strong because it seems like very few of these formats have pursued having an official media type, presumably for similar reasons to why HCL doesn't yet have one.

Thinking through the implications of the different major media types, I agree that application/hcl does feel like the best choice. text/hcl would suggest that it's valid to select a character encoding using the charset argument, but HCL is specified as always being UTF-8 encoded. I believe this is a similar rationale for JSON being represented as application/json rather than text/json.

---

This is not directly related to your question, but since you linked to gomplate I'd note that HCL (at least, in its current HCL 2 form) is unlike some of the other input formats gomplate supports in that it doesn't decode directly into a JSON-infoset-compatible data structure. Instead, it uses a schema given by the caller to interpret the data.

For that reason, it might not be straightforward to incorporate HCL into gomplate: you'll need to either allow the user to somehow provider a schema (one existing answer for that is the specification format associated with the hcldec command line tool) or define your own custom mapping from the HCL 2 infoset to something the Go template engine can consume.

The latter would be possible if you work only with the HCL native syntax, because its grammar is self-describing enough to be unambiguous, but the JSON HCL variant is ambiguous and relies on schema for correct decoding. (Perhaps you don't care about JSON HCL because gomplate already supports JSON directly anyway.)

If you mainly care about HCL as a syntax for writing down key/value pairs where the values can have data types similar to JSON then one further option is to use the dynamic attributes processing model, which gives a more restrictive form of HCL that only includes the attribute syntax and can operate without a schema. That means it will accept arbitrary attributes, like this:

foo = {
  bar = {
    baz = "qux"
  }
}

...but it will not accept the HCL block syntax, like this:

example "bar" {
  # ...
}

...because there are lots of different ways to map blocks into a JSON-like data structure, depending on the goals of the specific format that's being designed in terms of HCL, and so application code typically uses an application-specific schema and application-specific decoding logic for that.

[hairyhenderson]

[...] was amused to find the very similar toml-lang/toml#465!

😂 I never really expected that an IANA registration would result from that one, but the follow-up issue's still open, so there's always hope 😉

Thanks for the excellent info about schemas/etc - this is great clarification as I don't think I fully grasped the ramifications... I was working under some wrong assumptions!

I'll need to think about the best way to work with this in gomplate. I'm not sure the dynamic attributes processing model will be enough as I would presume most people wanting to parse HCL with gomplate will be interested in configs related to existing tools like Terraform or Consul.

In fact, the user who filed the feature request is mainly interested in reading Terraform 0.12 configs. I suppose I'll start by looking to specifically support the TF schema.

[radeksimko]

Related to what @apparentlymart said above about schema and the use case mentioned by @hairyhenderson I wonder if the schema is significant enough that it should be expressed in the mime-type too, e.g.

• application/hcl (schema-less HCL)
• application/hcl+terraform
• application/hcl+packer
• application/hcl+consul
• application/hcl+nomad
• application/hcl+vault

---

We will soon need to address a similar problem in the LSP where a language server processing configurations needs to be able to identify the language of the configuration, so it knows how to provide completion etc.

With that in mind we may even need to go as far as recognizing some limited syntax versions, such as tfvars - but I'm not sure if this is important at all in the context of mime types. It is in the LSP however.

So perhaps for Terraform we'd need two:

• application/hcl+terraform
• application/hcl+tfvars

Related: hashicorp/terraform-ls#50 (comment)

With all that said Wikipedia suggests that

Unregistered suffixes should not be used (since January 2013)

which means we'd probably need to register each suffix/schema individually. Maybe suffixes is not even the right way to express this? I'm not sure - we'd need to read the rules around this in more depth.

[hairyhenderson]

Thanks @radeksimko - I had a similar thought with respect to suffixes... There is precedent with XML with ~400 application/<foo>+xml types registered.

The convention is defined in RFC 6838 §4.2.8, and in this case would be application/terraform+hcl/application/tfvars+hcl/etc...

Personally I think it makes the most sense to first register application/hcl, and then follow up afterwards with others as necessary.

[radeksimko]

Personally I think it makes the most sense to first register application/hcl, and then follow up afterwards with others as necessary.

Generally we tend to discuss the implications of proposals like this within wider HashiCorp engineering org so I assume if we were to submit an RFC adding a new mime type it would undergo at least some internal scrutiny first, which may include discussing use cases and trade-offs between the schema-less HCL type and the schema-specific types.

If you really need a mime-type now, perhaps we could somehow leverage the x. prefix? I'm not sure of the implications in case of gomplate and what the potential migration strategy would be after we do register some mime types, but AFAICT such prefixed types don't require/assume IANA registration, so that might be a clean short-term solution?

[hairyhenderson]

If you really need a mime-type now, perhaps we could somehow leverage the x. prefix?

I don't need it immediately - I don't expect to have the time to implement HCL support for a little while anyway. But I hadn't considered using the x. tree; if it's unlikely for HashiCorp to register application/hcl, perhaps I can start with application/x.hcl. Obviously I'd rather use something more likely to be registered from the get-go.

Let me know if this is something that's going to be put on your internal roadmap!