From 7d8bd4d3f1e4033a67223ed6380fd2af74f749c5 Mon Sep 17 00:00:00 2001 From: Merel Theisen Date: Tue, 24 Jan 2023 14:21:07 +0000 Subject: [PATCH 1/2] Add docs about using env vars for credentials with omegaconfloader Signed-off-by: Merel Theisen --- docs/source/kedro_project_setup/configuration.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/source/kedro_project_setup/configuration.md b/docs/source/kedro_project_setup/configuration.md index 3e32969013..33053dc1a5 100644 --- a/docs/source/kedro_project_setup/configuration.md +++ b/docs/source/kedro_project_setup/configuration.md @@ -273,6 +273,19 @@ data: Since both of the file names (`parameters.yml` and `parameters_globals.yml`) match the config pattern for parameters, the `OmegaConfLoader` will load the files and resolve the placeholders correctly. +### Environment variables for credentials +The [`OmegaConfLoader`](/kedro.config.OmegaConfLoader) enables you to load credentials from environment variables. To achieve this you have to use the `omegaconf` [`oc.env` resolver](https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html#oc-env). +This is an example of how you can access credentials from environment variables in `credentials.yml`: + +```yaml +dev_s3: + client_kwargs: + aws_access_key_id: ${oc.env:AWS_ACCESS_KEY_ID} + aws_secret_access_key: ${oc.env:AWS_SECRET_ACCESS_KEY} +``` + +Note that you can only use the resolver in `credentials.yml` and not in catalog or parameter files. This is because we do not encourage the usage of environment variables for anything other than credentials. + ## Parameters ### Load parameters From 5bd4634064ca4af82c67626be83e9ab8457a0adc Mon Sep 17 00:00:00 2001 From: Merel Theisen Date: Wed, 25 Jan 2023 11:27:43 +0000 Subject: [PATCH 2/2] Format note correctly Signed-off-by: Merel Theisen --- docs/source/kedro_project_setup/configuration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/kedro_project_setup/configuration.md b/docs/source/kedro_project_setup/configuration.md index 33053dc1a5..e956a5f306 100644 --- a/docs/source/kedro_project_setup/configuration.md +++ b/docs/source/kedro_project_setup/configuration.md @@ -284,7 +284,9 @@ dev_s3: aws_secret_access_key: ${oc.env:AWS_SECRET_ACCESS_KEY} ``` +```{note} Note that you can only use the resolver in `credentials.yml` and not in catalog or parameter files. This is because we do not encourage the usage of environment variables for anything other than credentials. +``` ## Parameters