Skip to content

Commit

Permalink
docs: clarify requirements for Consul token policies and TTLs (#24167)
Browse files Browse the repository at this point in the history
As of #24166, Nomad agents will use their own token to deregister services and
checks from Consul. This returns the deregistration path to the pre-Workload
Identity workflow. Expand the documentation to make clear why certain ACL
policies are required for clients.

Additionally, we did not explicitly call out that auth methods should not set an
expiration on Consul tokens. Nomad does not have a facility to refresh these
tokens if they expire. Even if Nomad could, there's no way to re-inject them
into Envoy sidecars for Consul Service Mesh without recreating the task anyways,
which is what happens today. Warn users that they should not set an expiration.

Closes: #20185 (wontfix)
Ref: https://hashicorp.atlassian.net/browse/NET-10262
  • Loading branch information
tgross authored Oct 11, 2024
1 parent 373aae7 commit 7381f84
Show file tree
Hide file tree
Showing 2 changed files with 63 additions and 4 deletions.
60 changes: 57 additions & 3 deletions website/content/docs/configuration/consul.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,10 @@ agents.

- `token` `(string: "")` - Specifies the token used to provide a per-request ACL
token. This option overrides the Consul Agent's default token. If the token is
not set here or on the Consul agent, it will default to Consul's anonymous policy,
which may or may not allow writes. Will default to the `CONSUL_HTTP_TOKEN`
environment variable if set.
not set here or on the Consul agent, it defaults to Consul's anonymous policy,
which may or may not allow writes. Defaults to the `CONSUL_HTTP_TOKEN`
environment variable if set. Nomad cannot refresh this token; if the token is
deleted, Nomad is not able to communicate with Consul.

- `verify_ssl` `(bool: true)`- Specifies if SSL peer verification should be used
when communicating to the Consul API client over HTTPS. Will default to the
Expand Down Expand Up @@ -334,6 +335,58 @@ consul {
}
```

### Consul ACL policy for Nomad

Nomad agents need access to Consul in order to register themselves in the
service catalog and discover other Nomad agents via service discovery for
automatic clustering. Nomad clients use Consul tokens from Workload Identity to
register services and checks but need permissions on their own token to
deregister. Nomad servers also create [configuration
entries][consul_config_entry] for Consul Service Mesh, so the specific
permissions vary slightly between Nomad servers and clients. The following
Consul ACL policies represent the minimal permissions Nomad servers and clients
need.

<Tabs>
<Tab heading="Nomad Servers">

```hcl
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "write"
}
service_prefix "" {
policy = "write"
}
acl = "write"
mesh = "write"
```

</Tab>
<Tab heading="Nomad Clients">

```hcl
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "write"
}
service_prefix "" {
policy = "write"
}
```

</Tab>
</Tabs>

### Consul Namespace <EnterpriseAlert inline/>

Consul does not allow ACL policies associated with namespaces to use agent
Expand Down Expand Up @@ -383,3 +436,4 @@ namespace "nomad-ns" {
[taskuser]: /nomad/docs/job-specification/task#user "Nomad task Block"
[consul_bound_aud]: /consul/docs/security/acl/auth-methods/jwt#boundaudiences
[`consul.namespace`]: /nomad/docs/job-specification/consul#namespace
[consul_config_entry]: /consul/docs/connect/config-entries
7 changes: 6 additions & 1 deletion website/content/docs/integrations/consul/acl.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ running in Nomad.

Nomad agents need access to Consul in order to register themselves in the
service catalog and discover other Nomad agents via service discovery for
automatic clustering. Nomad servers also create [configuration
automatic clustering. Nomad clients use Consul tokens from Workload Identity to
register services and checks but need permissions on their own token to
deregister. Nomad servers also create [configuration
entries][consul_config_entry] for Consul Service Mesh, so the specific
permissions vary slightly between Nomad servers and clients. The following
Consul ACL policies represent the minimal permissions Nomad servers and clients
Expand Down Expand Up @@ -106,6 +108,9 @@ URL][nomad_jwks_url]. Consul servers call this URL to retrieve the public keys
Nomad uses to sign workload identities. With these keys, Consul is able to
validate their origin and confirm that they were actually created by Nomad.

Nomad cannot recreate Consul tokens that have been deleted. The auth method
configuration should never set the `MaxTokenTTL` field.

<CodeBlockConfig highlight="2" filename="auth-method.json">

```json
Expand Down

0 comments on commit 7381f84

Please sign in to comment.