Unable to load context in jupyter notebook: error in function _parse_requirements

[vchaparro]

Hi,
I have a problem when running kedro jupyter notebook. The context variable is not loaded. When I do %reload_kedro in my notebook, I see this error:

c:\users\quark.conda\envs\tfm-env\lib\site-packages\kedro_mlflow\utils.py in _parse_requirements(path, encoding)
71
72 def _parse_requirements(path, encoding="utf-8"):
---> 73 with open(path, mode="r", encoding=encoding) as file_handler:
74 requirements = [
75 x.strip() for x in file_handler if x.strip() and not x.startswith("-r")

FileNotFoundError: [Errno 2] No such file or directory: 'src\requirements.txt'

However, the requirements file is there. I needed to modify that function to do nothing and I got the context variable available again.

Thanks in adavance!

[Galileo-Galilei]

Hello, I think this error is both easy to solve for your use case... but cannot be totally fixed with kedro<=0.16.5. Please tell me if one of below solutions works for you.

On the short term

If you use pipeline_ml in your code

You can very likely fix this bug by giving an absolute path (rather than relative src/requirements.txt) in the call of MlflowPipelineHook() in your run.py file.

If you do not use pipeline_ml in your code

You can very likely fix this bug by giving an empty dict in the call of MlflowPipelineHook() in your run.py file.

If you are not using this package for production

You an try out the develop version of the package: pip install --upgrade git+https://github.com/Galileo-Galilei/kedro-mlflow@develop, remove the arguments from in the call of MlflowNodeHook() and MlflowPipelineHook() in your run.py file, and (eventually) add an absolute path to your pipeline_ml calls.

pipeline_ml(training=YOUR_TRAINING_PIPELINE,
                  inference=YOUR_PREDICTION_PIPELINE, 
                  conda_env="ABSOLUTE/PATH/TO/REQUIREMENTS" #add this line
)

On the long term

I strongly suspect that your error comes from #30. The kedro jupyter probably modifies the working directory and the call is not made at the root of your kedro prpoject which causes the error. I plan to open an issue on kedro repo to suggest them somes changes on hook management, but we will be dependent on their development + design decisions will likely take times (if it is ever implemented).

[vchaparro]

As currently I'm not using pipeline_ml, giving an empy dict in the call works for me.
Thank you @Galileo-Galilei .