Pull Consul credentials from Vault

[rwe]

consul-template can currently retrieve and refresh secrets from Vault, including consul credentials. This is extremely useful, especially because consul-template handles automatically renewing those leases.

However there's no mechanism to use consul credentials from Vault within consul-template itself, short of nesting consul-template within itself, or having consul-template generate a configuration file in its own configuration path—both of which seems a little eyebrow-raising.

This would be a useful feature because it would allow consul-template to bootstrap itself with only a Vault token. Right now, systems with restrictive ACLs need to first get a consul token from Vault before starting consul-template, and then can use consul-template for the rest of the credentials. That's a lot of duplicated effort.

Something like:

token_vault_path = "/consul/creds/foo-service"
# As an alternative to
# token = "abcd1234"

[sethvargo]

Hi @erydo

Thank you for opening an issue. I'm a bit confused. You want Consul Template to retrieve its Consul token from Vault, then use that token for Consul requests?

[rwe]

@sethvargo - Yes, that's correct. If you start with a Vault token, this functionality would allow Consul Template to act as the main entry point to all protected configuration allowed by that token, and puts Vault secrets/creds and ACL-protected Consul keys on similar footing, configuration-wise.

Without this functionality, if using Vault to manage Consul credentials, it's necessary to add a script to first retrieve a Consul token from Vault for Consul Template to use (via vault read consul/creds/foo-service), and also to handle refreshing that token's credential lease. This is a direct duplicate of Consul Template's built-in functionality. Consul Template already does a good job with handling wrapped vault tokens and refreshing those and other credential leases.

[russroy]

Or ... in the same vein, why not take it a step further back still and allow consul-template to consume a Vault wrapped secret_id + role_id.

secret_id --> Vault token --> Consul token

This would be nirvana for services using approle auth and Consul dynamic secrets in Vault :-)

[rwe]

@russroy I feel that approach would be over-constrained to a specific mechanism for retrieving that Vault token. For example, we use EC2 auth rather than App Roles. Starting with a Vault token—however it's gotten—is, I feel, a very general point of entry.

[russroy]

@erydo I take your point. But if allowing both means of entry wouldn't be considered too messy I'd be in favor of that.

[sethvargo]

@russroy you can already give it a wrapped token

[russroy]

@sethvargo Correct me if I'm wrong but a wrapped secret_id is not interchangeable with a wrapped vault token.

[russroy]

To quote Jeff Mitchell

The idea isn't to give your services a Vault token to allow them to
generate secret-ids; the idea is you give them the secret-id instead.
(You can also use the app-id model of pushing your own secret-ids in.)

The main benefit you have over giving out tokens is that when a token
fetched with an approle secret-id expires, the secret-id (unless also
bound by an expiry or use-count) can be used to get a new token.
Otherwise you have to give your application a new token somehow.

[https://groups.google.com/forum/#!searchin/vault-tool/response$20wrapping$20app$20role%7Csort:relevance/vault-tool/w9HmAoT4d1U/KnRduSeqCAAJ]

For apps using Vault approle, Consul dynamic secrets, and CT the

secret_id --> Vault token step and
Vault token --> to Consul token step

needs to happen somewhere where they can be renewed - using the secret_id when necessary. To me, it make sense to factor it into CT (though I agree w/ @erydo that accommodating non-approle users is important too.)

[sethvargo]

Hi @russroy

This isn't the proper thread to have this discussion. Please create a new issue if this is something you're interested in pursuing. Thanks!

[russroy]

Ok, I understand. Will do.

[criloz]

@sethvargo I think that a possible way to do this is to implement it directly in the consul/api DefaultConfig function, if the needed env variables are present (like VAULT_TOKEN, VAULT_CONSUL_CREDENTIAL_PATH, VAULT_HOST, etc.) then read the token from vault.

and in each function that make a request verify if is time to renew the token if is not defined in the Query or Write Options. this will enable to each consul client based in the consul/api to have this feature.

just a thought

[zakcrawford]

@sethvargo Has this feature been put on the roadmap? Would a PR be considered that added this functionality?

[sethvargo]

Hi all,

Thank you for opening this issue and participating in the conversation. This is not possible in the short or long term given Consul Template's architecture, unfortunately. Consul Template requires it has all the connections to upstream APIs (Consul, Vault, etc) before it can start servicing requests. This creates a chicken-and-egg problem for which there is currently no solution in the current architecture. One could also argue that they want to pull a Vault token from somewhere in Consul's KV store. As such, this is not something that is currently on our roadmap or is it something we plan to implement ourselves. We would gladly review a Pull Request for this functionality, but I am going to close out this issue to prevent any false hope. Thanks and sorry!