Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create KedroMlflowConfig.setup() method and enable configuration to export credentials as environment variables #126

Merged
merged 1 commit into from
Nov 27, 2020

Conversation

Galileo-Galilei
Copy link
Owner

Description

Fix #31 and #97.

Development notes

  • When instantiating a KedroMlflowConfig object, there is no interaction with mlflow to create the associated experiment. It must be created explicitly
  • A method was added to export mlflow credentials on the fly when running a pipeline.
# credentials.yml
my_credentials: 
  AWS_SECRET_KEY: <key>
# mlflow.yml
credentials: my_credentials
  • interactively, you must now call the setup method to setus the configuration up:
# exploratory.py
from kedro.context import load_context
from kedro_mlflow.context import get_mlflow_config

context=load_context(".")
mlflow_config=get_mlflow_config(context)
mlflow_config.setup(context)

Checklist

  • Read the contributing guidelines
  • Open this PR as a 'Draft Pull Request' if it is work-in-progress
  • Update the documentation to reflect the code changes
  • Add a description of this change and add your name to the list of supporting contributions in the CHANGELOG.md file. Please respect Keep a Changelog guidelines.
  • Add tests to cover your changes

Notice

  • I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

  • I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.

  • I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.

  • I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

@codecov-io
Copy link

codecov-io commented Nov 26, 2020

Codecov Report

Merging #126 (c48c869) into develop (63dcd50) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##           develop      #126   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           25        25           
  Lines          713       725   +12     
=========================================
+ Hits           713       725   +12     
Impacted Files Coverage Δ
kedro_mlflow/framework/context/config.py 100.00% <100.00%> (ø)
kedro_mlflow/framework/hooks/pipeline_hook.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63dcd50...c48c869. Read the comment docs.

@Galileo-Galilei Galileo-Galilei force-pushed the feature/mlflow-credentials branch from 70760cc to c48c869 Compare November 26, 2020 20:01
@Galileo-Galilei Galileo-Galilei force-pushed the feature/mlflow-credentials branch from c48c869 to 27a820e Compare November 26, 2020 21:30
Copy link
Collaborator

@takikadiri takikadiri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great ! we are now consistent with kedro in creds management. 👍
We have to do the same for our datasets that have run_id attributes (they already have credentials entry). Maybe in another PR

@Galileo-Galilei
Copy link
Owner Author

Galileo-Galilei commented Nov 27, 2020

Managing datasets credentials is a totally different thing because we need to export temporarily credentials to avoid interfering with the tracking credentials. This is in the very short term todo list, though.

@Galileo-Galilei Galileo-Galilei linked an issue Nov 27, 2020 that may be closed by this pull request
@Galileo-Galilei Galileo-Galilei merged commit 9239a5a into develop Nov 27, 2020
@Galileo-Galilei Galileo-Galilei deleted the feature/mlflow-credentials branch November 27, 2020 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants