Implement idiomatic environment variable handling

[rocketraman]

Leverage the capability exposed by #413 to implement idiomatic environment variable handling.

Fixes #410.

I've tried to implement this in a backward compatible way, but if we are ok with breaking compatibility for 3.0, given the new path normalization capability, we could replace the existing logic for env vars and simplify it down to always apply env vars with the new logic, removing the EnvironmentVariableOverridePropertySource and simplifying EnvironmentVariablesPropertySource to remove all the config vals, which is what I understood from #413 (comment).

This PR is stacked on top of #413 and will be rebased and moved out of draft once that PR is merged.

There is a test failure in this PR which will be resolved once #444 is merged.

[rocketraman]

@sksamuel Now that #413 is merged (thank you!), I have rebased this.

[rocketraman]

@sksamuel Any update on this?

[sksamuel]

Happy for this to be breaking now, and we'll cut 3.0

[rocketraman]

@sksamuel Ok, then here is what I propose (this is what would be in the updated README)... please review and let me know if you are ok with this, and if you are I will make the appropriate changes.

---

EnvironmentVariablesPropertySource

The EnvironmentVariablesPropertySource reads config from environment variables. This property source maps environment variable names to config properties via idiomatic conventions for environment variables. Env vars are idiomatically UPPERCASE and contain only letters (A to Z), digits (0 to 9), and the underscore (_) character.

Hoplite maps env vars as follows:

• Underscores are separators for nested config. For example TOPIC_NAME would override a property name located in a topic parent.

• To bind env vars to arrays or lists, postfix with an index e.g. set env vars TOPIC_NAME_0 and TOPIC_NAME_1 to set two values for the name list property.

• To bind env vars to maps, the key is part of the nested config e.g. TOPIC_NAME_FOO and TOPIC_NAME_BAR would set the "foo" and "bar" keys for the name map property. Note that keys are one exception to the idiomatic uppercase rule -- the env var name determines the case of the map key.

If the optional (not specified by default) prefix setting is provided, then only env vars that begin with the prefix are considered, and the prefix is stripped from the env var before processing.

As of Hoplite 3, the EnvironmentVariablesPropertySource is applied by default and may be used to override other config properties directly. There is no longer any built-in support for the config.override. prefix. However, the optional prefix setting can still be used for the same purpose.

[rocketraman]

@sksamuel I've rebased this branch on the latest master, and in line with my previous message I've pushed the changes to implement the breaking but simpler (i.e. less configurable, more idiomatic) but overall more capable EnvironmentVariablesPropertySource.

Most of the changes to unrelated tests were because the defaults have changed, and now include the default environment.

I've also added support for Spring-like env var to array/list configuration, which should resolve #402.

[rocketraman]

@sksamuel bump

[sksamuel]

Looks good, I guess if you want to use a prefix, you have to remove all the default sources, and add them all back in, but with a custom EnvironmentVariablesPropertySource.
Maybe we should just not have env registered by default anymore?

[rocketraman]

Looks good, I guess if you want to use a prefix, you have to remove all the default sources, and add them all back in, but with a custom EnvironmentVariablesPropertySource.

@sksamuel My thinking was that if people want to customize property sources, including the EnvironmentVariablesPropertySource then the normal pattern would be to initialize HopLite via ConfigLoaderBuilder.defaultWithoutPropertySources(), and then add in the customized property sources manually.

There is also the ConfigLoaderBuilder.emptyByDefaultPropertySources() which gets the user all the defaults except the EnvironmentVariablesPropertySource, so users could do something like:

ConfigLoaderBuilder
  .defaultWithoutPropertySources()
  .addPropertySource(EnvironmentVariablesPropertySource(prefix = myPrefix))
  .emptyByDefaultPropertySources()

We could potentially make the pattern simpler with some more public funs on ConfigLoaderBuilder, maybe something like defaultWithEnvVarPrefix(prefix: String) and defaultPropertySourcesWithEnvVarPrefix(prefix: String).

Maybe we should just not have env registered by default anymore?

That is an option as well. I do think with the popularity of 12-factor apps, env vars are now very commonly used for config overrides (and rightly so), so I would hesitate to remove it from the defaults.