Support for Vault approle workflow in consul-template.

[russroy]

This is a request for a consul-template enhancement to better support the Vault approle workflow.

I see consul-template as the "glue" between Vault/Consul and services which consume their secrets and discovery services. At present, services which wish to make use of both Vault's AppRole authentication backend and its Consul dynamic secret backend cannot to do so seamlessly without some enhancements to CT. (Actually, some, but not all, of this is being addressed here issue #733.)

Ref's

1. https://www.hashicorp.com/blog/vault-0.6.html#response-wrapping
2. https://www.vaultproject.io/docs/auth/approle.html
3. https://www.vaultproject.io/docs/secrets/consul/index.htm
4. https://groups.google.com/forum/#!searchin/vault-tool/response$20wrapping$20app$20role%7Csort:relevance/vault-tool/w9HmAoT4d1U/KnRduSeqCAAJ

Here's a proposed workflow w/ ref's to existing HC workflow descriptions:

1. Some (unnamed) entity (a Vault client, outside of CT) which is responsible for deploying a trusted service foo in an environment will request of Vault a wrapped secret_id issued against a prefined approle we wish to associate with the service.

   $ vault write -f auth/approle/role/testrole/secret-id -wrap-ttl=20s
   

   (Essentially steps 1-3 of ref.1)

2. The client, as part of the app deployment process, instantiates a consul-template daemon passing on

   • the wrapped (single-use, time restricted) Vault secret_id from step 1
   • an associated (non-secret) role-id
   • and the desired Consul policy name (this last part is the subject of issue#733).

   (Essentially step 4 of ref.1)

   This is in lieu of specifying vault_token and consul_token CT config parameters.

3. CT unwraps the secret_id

   resp=$(curl -s -XPOST -H "X-Vault-Token:$wrapped_secret_id" "$vault_addr/v1/sys/wrapping/unwrap")
   secret_id=$(echo $resp | jq -r .data.secret_id)
   

   (Step 5&6 of Ref 1)

4. CT logs into Vault w/ secret_id + role_id to obtain Vault token

   resp=$(curl -s -XPOST "$vault_addr/v1/auth/approle/login" -d@- <<EOF
   {
   "role_id": "$role_id",
   "secret_id": "$secret_id"
   }
   EOF)
   
   vault_token=$(echo $resp | jq -r .auth.client_token)
   resp=$(curl -s -XPOST "$vault_addr/v1/auth/approle/login" -d@- <<EOF
   

   (Ref2)

5. CT uses Vault token + Consul policy to obtain Consul token

   policy=consul/creds/readonly
   resp=$(curl -s -XGET -H "X-Vault-Token: $vault_token" "$vault_addr/v1/$policy") consul_token=$(echo $resp | jq -r .data.token)
   

   (Ref3)

CT should now

• renew Vault and Consul token leases as necessary
• In the event of some temporary communication failure whereby a token expires, CT should use
  secret_id (unless also bound by an expiry or use-count) to obtain a new Vault token
  (See ref 4)

Or something like that...

To do all this outside the CT engine would entail replicating the lease renewal stuff which is already part of CT.

[jantman]

For what it's worth, I don't think this belongs in consul-template. I know that supporting AppRole has become popular among some third-party tools, but it's only one of many authentication methods. For tools like consul-template, I think the right thing is to support reading a token from a file.

The fact is, there's really a glaring hole in the Vault tooling ecosystem right now. For Vault Enterprise customers running on EC2 Instances, there's the Vault Secure Introduction Client which manages authenticating to Vault with EC2 Auth, writing the token out to a file on disk, and renewing the token/re-authenticating as necessary. It seems to me that there's a missing tool in the ecosystem which would perform this same purpose for "any" auth method (and ideally be open source)...

[jsoriano]

@jantman we have started to play a bit with vault and approles to try to properly automate secrets provisioning in our config management and we created approle-login, a small tool that does just that, it logins with approle and writes a token to a file so other tools can use it.

[ctlajoie]

I think all that is needed to support this workflow is the ability to set the X-Vault-Wrap-TTL header when making the request to Vault. This could be done using a special key name (like x-wrap-ttl) that would cause CT to add the header instead of adding it to the json payload. For example:

{{ with secret "auth/approle/role/testrole/secret-id" "role_name=testrole" "x-wrap-ttl=5m" }}
VAULT_TOKEN={{ .Data.wrapping_token }}
{{ end }}

Thoughts?

[jsoriano]

@ctlajoie this is not only about wrapping secrets. As @russroy described, the secrets consumer would need to unwrap a secret ID (that was previously wrapped by another entity) and then use this secret ID with a role ID to request a token. As @jantman said I don't now if it worths to implement all vault auth mechanisms in consul-template.

[pearkes]

Hey all. In trying to understand the root ask here I think I came back @jsoriano's point here:

The fact is, there's really a glaring hole in the Vault tooling ecosystem right now. For Vault Enterprise customers running on EC2 Instances, there's the Vault Secure Introduction Client which manages authenticating to Vault with EC2 Auth, writing the token out to a file on disk, and renewing the token/re-authenticating as necessary. It seems to me that there's a missing tool in the ecosystem which would perform this same purpose for "any" auth method (and ideally be open source)...

We definitely agree, obviously this issue in question is from 2017 so we are late to respond here, but I think the introduction problem can largely be solved in conjunction with the Vault agent and auto-auth functionality, which was in the latest Vault release:

https://www.vaultproject.io/docs/agent/autoauth/index.html

This can be used to authenticate in a wide variety of environments and introduce a token to consul-template (or other tooling) which can be used to either retrieve a Consul ACL token via Vault or read other secrets from Vault. This is not directly integrated with consul-template, so the introduction would need to be out of band.

See also #1128 (comment).

I may be off here in understanding the request, so please comment back as such and we can continue the discussion.

[adawalli]

As far as I can tell, vault agent autoauth doesn't support vault namespaces. Consul-agent doesn't support approle method. It doesn't really seem like there is a clear path forward besides doing everything through curl commands (if you use approle and namespaces).

[catsby]

Hey @adawalli - Vault Agent does support Vault namespaces, it's a field common to the auth block:

• https://www.vaultproject.io/docs/agent/autoauth/index.html#configuration-method

example:

auto_auth {
  method "aws" {
    mount_path = "auth/aws-subaccount"
    namespace = "some_vault_namespace"
    config = {
      type = "iam"
      role = "foobar"
    }
  }
[ ... ]
}

[harningt]

Another benefit for consul-template supporting the approle workflow would be the memory footprint should be a lot smaller. If I recall right - the memory used by vault running as an agent is a lot larger than consul-template uses (80MB vs 12MB). Sure small price to pay for simplicity - but scaling it out it can add up.