Vault agent listener without cache silently ignored

[vvidovic]

When you configure vault agent with listener stanza but without cache stanza listener is not turned on but silently ignored.

It would be nice to either:

• vault agent would fail to start with a message like "The listener is declared without cache configuration, aborting."
• vault agent would start with "proxy" listener without caching
  • it would still be useful because an agent's auto-auth would simplify client application usage of the Vault
  • a sidecar agent container can be configured to securely connect to Vault server and expose non-secure vault access on localhost/Unix socket

Here is example configuration from OpenShift environment using k8s auto_auth:

    vault {
      address = "https://vault-poc-svc-1.vault-poc.svc:8200"
      ca_cert = "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt"
    }

    auto_auth {
      method "kubernetes" {
        config = {
          role = "nvda"
        }
      }

      sink "file" {
        config = {
          path = "/appconfig/vault_token"
        }
      }
    }

    cache {
      use_auto_auth_token = true
    }

    listener "tcp" {
      address = "127.0.0.1:8100"
      tls_disable = true
    }

    template {
      source      = "/agentconfig/vault-agent-secret.tmpl"
      destination = "/appconfig/secret.json"
    }

[hamishforbes]

It would be very useful for us to be able to disable the caching behaviour without losing the proxy / auto-auth token injection features.

Agent caching seems to be interfering with my usage of consul-template as they're both trying to renew and I think i'm running into a race of some kind.
Where consul-template knows the secret (DB creds in this case) is expiring, tries to read a new value but gets back a cached response from the agent with credentials which are about to expire.
Which also means the rendered template doesn't change and the child process doesn't get restarted.

I haven't been able to pin it down but there must be other scenarios where caching isn't desirable.

[lneva-fastly]

@hamishforbes I'm seeing exactly the same problem you are with consul-template. Currently trying to figure out a solution. Did you solve this?

[briantist]

Adding another vote for this.

First, it was extremely confusing that I was defining a listener, and there was no error, but also no listener.

Second, I am wanting to set up Vault agent specifically as a proxy, with auto-auth (a request can be made unauthenticated to the agent, and it will automatically use the token that it's been maintaining via auto-auth), but without caching. I want this for kv2 secrets which can't be cached anyway, even if I wanted them to (I might be ok with it later with more consideration).

As it is now, I can turn "on" caching with auto-auth, and have it work the way I want, but the problem is that if you ever decide to suddenly support caching kv2, we'll automatically start caching them by upgrading our Vault client, even without us changing configuration.

So I think it would be much better to add a separate proxy use case, where we can have this functionality without needing to configure caching.

[iiro]

And another vote from here.

The current behaviour with dropping the cache-stanza while leaving the listener leaving without any listener is very confusing. An update needed here, at least logging an error (if this is intentional atm)...

Also, regarding the caching: it's kinda understandable that caching the KV2's not supported at the moment - but I seriosly hoping this would change in a near future. Maybe - as the agent cannot follow a TTL to expire them from the cache - you could just introduce a setting in Agent config's cache on how long you'd cache them (if you want to do this at all)...?

In our case, most of the apps use both - dynamic and static credentials. So, the current cache functionality works kind of half-way only - as it does not properly protect from any Vault connectivity problems...

[jardinezm]

This has been added submitted as a feature request for visibility and tracking.

[kauedg]

This behavior should at least be mentioned in the agent's documentation.

Something like

listener (listener: <optional>) - Specifies the addresses and ports on which the Agent will respond to requests. Ignored if a 'cache' configuration block is not declared. 

[F30]

Contributing to my confusion around this was the fact that even the example config for Auto-Auth contains a (apparently meaningless) listener block.

[heatherezell]

Tagging our documentation expert @taoism4504 here. I will work with her to see if we can improve the documentation on this. Thanks for the feedback! :)

[foobarmeow]

Ran into this and this issue was the only thing that helped me resolve it. Wanted to add my voice to users who'd like the documentation to reflect this. ✌️

[finnstech]

I've added a PR to update the docs. Thanks for the feedback!

I am the PM working on Agent and I'm curious about the use case for setting up Agent like this as described by @hamishforbes and @vvidovic -- it's like you're turning Agent into a secure tube to pass secrets through to K8s from Vault server?

[hamishforbes]

I think my original comment on this issue predates my usage of Kubernetes, but yes essentially.

My application container uses consul-template in exec mode to render Vault secrets to a file and then supervisor the application process.
This is so the application itself doesn't need to implement watching a file(s) for changes and reloading config.
Consul-template handles that and signals the app process when something changes (in my case dynamic database credentials being refreshed as they near the end of their lease time).

Using the Vault agent as a secure proxy simplifies the consul-template configuration as it just needs to be pointed at an unauthenticated URL, rather than needing Vault TLS ca/certs and tokens etc etc

This also well predates the ability of consul-template to use Vault Kubernetes auth natively.

[briantist]

@finnstech my use case also has nothing to do with kubernetes. It's just about having Agent act as a proxy so that clients can make unauthenticated requests and have the Agent inject the auto-auth token in its request to the upstream Vault server.

[VioletHynes]

Hi there! Going to close this issue as we've just implemented this as part of #18137 - this will be available in the next major release.

In short, you'll be able to use a listener stanza (and the API Proxy) irrespective of if a cache block is specified. The docs have also been clarified significantly.

Thanks for raising this issue!