Support for secrets set in ENV variables

[kaihendry]

Currently I don't see any support for SMTP_AUTH_PASSWORD to be set via typically Docker env variables. Am I mistaken?

[brian-brazil]

The chosen approach is to put them in the config file.

There's many many possible ways to provide configuration, for sanity we have to choose just one of them.

[kaihendry]

It's fairly common practice now to provide secrets via ENV variables & it would be good if you could support this approach together with your configuration. Ideally your configuration can be set by an ENV variable like shell.

[brian-brazil]

You're free to put scripts around our configs to provide this for yourself.

[kaihendry]

But that would mean forking the Docker image, right? So I can add a little m4 substitution or something.

[Andor]

@kaihendry you can mount directory with your config into container with alertmanager.

[dolan-a]

@brian-brazil -- Is there a thread somewhere that has the discussion on ending support for environment variables? I'm trying to understand why this design decision was made.

From a programming standpoint, it would be fairly trivial to modify config/config.go to loop over all configuration attributes, looking for a matching environment variable, and overwriting the stored value if one is found. (FWIW, Docker's source appears to have Go code for doing this.)

Given that many organizations store configuration files in source repositories (e.g., git) and that is a highly-recommended practice to not store secrets in source repositories, allowing prometheus/alertmanager users to store secrets in environment variables (without having to use awk hacks) seems like a great idea.

Thank you in advance for any clarification you can provide.

[brian-brazil]

It was a matter of consistency and allowing for many many secrets to be provided.

We want to have just one way to configure a given setting, and users can then build on that.

[fvigotti]

If possible I would like to vote for some kind of env interpolation for config files, this is a common best practice and would be a great feature in prometheus alertmanager..

[seanhoughton]

If environment variables aren't going to be supported what about encrypted secrets in the config? The eyaml (https://github.com/TomPoulton/hiera-eyaml) project works well for Puppet.

[brian-brazil]

That is not something we plan on supporting. Once again, too many ways to do it. You're free to use your own tooling around this.

[groob]

I'd like to chime in and say that ENV variables should be supported as well. given that so many of us are using prometheus with docker/k8s deployment, it's very odd to have to deal with the config file and not have builtin support for interpolating secrets/env variables.

[shaneramey]

One option is to use a k8s init-container with confd to write your environment-variable-passed secrets to a config file (on a shared emptyDir: {} volumeMount) before the Alertmanager container starts. The Alertmanager container would then mount that same Volume to consume the populated template as its config file.

Supporting environment variables (such as SMTP_AUTH_PASSWORD) natively in the Alertmanager container would be difficult if users wanted multiple alert routing destinations of the same type (like 2+ SMTP servers with different credentials).

[brian-brazil]

One option is to use a k8s init-container with confd to write your environment-variable-passed secrets to a config file

That's one option.

[brust]

@brian-brazil Sorry, I didn't understand.

How can I get a env-var to use it on my config.yml? What's the options?

My config.yml is versioned and I don't want to put sensitive data there.