-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an oc.decode
resolver to parse and evaluate strings
#574
Milestone
Comments
This was referenced Feb 26, 2021
odelalleau
added a commit
to odelalleau/omegaconf
that referenced
this issue
Mar 15, 2021
* Restore and deprecate the old `env` resolver for backward compatibility with OmegaConf 2.0 * The new `oc.env` resolver keeps the string representation of environment variables, and does not use the cache * The new `oc.decode` resolver can be used to parse and evaluate strings according to the OmegaConf grammar Fixes omry#383 Fixes omry#573 Fixes omry#574
odelalleau
added a commit
to odelalleau/omegaconf
that referenced
this issue
Mar 15, 2021
* Restore and deprecate the old `env` resolver for backward compatibility with OmegaConf 2.0 * The new `oc.env` resolver keeps the string representation of environment variables, and does not use the cache * The new `oc.decode` resolver can be used to parse and evaluate strings according to the OmegaConf grammar Fixes omry#383 Fixes omry#573 Fixes omry#574
odelalleau
added a commit
to odelalleau/omegaconf
that referenced
this issue
Mar 18, 2021
* Restore and deprecate the old `env` resolver for backward compatibility with OmegaConf 2.0 * The new `oc.env` resolver keeps the string representation of environment variables, and does not use the cache * The new `oc.decode` resolver can be used to parse and evaluate strings according to the OmegaConf grammar Fixes omry#383 Fixes omry#573 Fixes omry#574
odelalleau
added a commit
that referenced
this issue
Mar 18, 2021
* Introduce new `oc.env` and `oc.decode` resolvers * Restore and deprecate the old `env` resolver for backward compatibility with OmegaConf 2.0 * The new `oc.env` resolver keeps the string representation of environment variables, does not use the cache, and accepts None as default value * The new `oc.decode` resolver can be used to parse and evaluate strings according to the OmegaConf grammar Fixes #383 Fixes #573 Fixes #574 * Allow `oc.decode` to evaluate interpolations * Improve documentation of `oc.decode` In particular it explains that dictionaries and lists are automatically converted to transient config nodes. * Update docs/source/usage.rst Co-authored-by: Omry Yadan <omry@fb.com> * Update news/573.api_change Co-authored-by: Omry Yadan <omry@fb.com> * Update omegaconf/_utils.py Co-authored-by: Omry Yadan <omry@fb.com> * Update omegaconf/_utils.py Co-authored-by: Omry Yadan <omry@fb.com> * Remove the USERID example * Show example of quoted string as default value for `oc.env` * Remove duplicated comments (# #) * Validate default value of `oc.env` even when not used * Simplify tests with recwarn * Use convenience `show()` function in doc to show type and value * Update doc on string interpolations * More readable test formatting * Improve comment formatting * Restore interpolation examples * Update docs/notebook/Tutorial.ipynb Co-authored-by: Omry Yadan <omry@fb.com> * Update docs/source/usage.rst Co-authored-by: Omry Yadan <omry@fb.com> * Update docs/source/usage.rst Co-authored-by: Omry Yadan <omry@fb.com> * Rephrasing in doc * Use `show()` function in doc * Raise a KeyError instead of ValidationError for missing env variables * Remove handling of "null" as default in legacy env resolver * Update news * Explicit typing for the default value of the `oc.env` resolver * Use a more appropriate exception type * Update tests/test_interpolation.py Co-authored-by: Omry Yadan <omry@fb.com> * Safer markers for default values * Fix coverage * Use more appropriate TypeError * Refactor: consistent use of _DEFAULT_MARKER_ Co-authored-by: Omry Yadan <omry@fb.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
With
env
being deprecated in favor ofoc.env
(#573), which always returns environment variables as strings, we need an easy way to convert strings to common types like bool, int, float, etc.Describe the solution you'd like
Introduce a new resolver
oc.decode
that converts a string automatically, using the common OmegaConf grammar.For instance:
should print:
In addition,
${oc.decode:null}
is valid and returnsNone
, in order to support for instance${oc.decode:${oc.env:DB_PORT,null}}
Differences with the old env resolver
Note that
${oc.decode:${oc.env:VAR}}
is not strictly equivalent to the old${env:VAR}
from OmegaConf 2.0. This is becauseoc.decode
relies on the same parsing logic as used when parsing resolver arguments in interpolations, whileenv
was using more basic custom parsing functions. Some key differences are:oc.decode
will raise an exception if the content of the environment variable is invalid according to the grammar (ex: an unquoted string containing invalid characters). One may use quotes (ex:VAR="'~!&*'"
) to avoid such exceptions (or just not useoc.decode
to keep the original content of the variable)oc.decode
handles more data types like dictionaries, lists, null and even interpolations, which would have been kept as strings when usingenv
The text was updated successfully, but these errors were encountered: