Kedro Newb Struggling To Get Catalog in Spaceflights tutorial working

[dwinski]

I'm new to kedro and running 0.17.6 on a Windows machine inside a conda environment with Python 3.8.12 where I have pip installed all the requirements from the requirements.txt file that was created with "kedro new". I also tried to install the pandas.CSVDataSet and pandas.ExcelDataSet classes by modifying the line in the requirements.txt file per the spaceflights tutorial to:
kedro[pandas.csvdataset,pandas.exceldataset]==0.17.6

I'm trying to follow the spaceflights tutorial but catalog in the kedro ipython session is not working (NameError: name 'catalog' is not defined) even when I try the magic commands below per this stackoverflow thread: https://stackoverflow.com/questions/70080915/kedro-context-and-catalog-missing-from-ipython-session

%load_ext kedro.extras.extensions.ipython
%reload_kedro <path_to_project_root>

However, when I run:
%reload_kedro <path_to_project_root>

I get an exception:
DataSetError: An exception occurred when parsing config for DataSet 'companies': Class 'pandas.CSVDataSet' not found or one of its dependencieshas not been installed.

Would really like to get through this tutorial. Any help would be appreciated.

[datajoely]

Hi @dwinski!

Did you run kedro install after modifying the requirements.txt? You can also skip this part and quickly run pip install "kedro[pandas.CSVDataSet, pandas.ExcelDataSet]"==0.17.6

That catalog is not defined error is because Kedro cannot create the catalog without the right requirements installed

[dwinski]

Ok, glad dependencies are correct. Are you able to reproduce the catalog error in kedro ipython in your spaceflights-dec15 environment?

[dwinski]

Btw, I was doing the spaceflights tutorial from scratch (did not use starter with "kedro new")

[datajoely]

Hi @dwinski I'm not exactly sure what's gone wrong here - but I've run through the tutorial until the point you've got stuck and pushed my progress to this repository:

https://github.com/datajoely/spaceflights-dwinski/

At this point I'm able to do the kedro ipython workflow. We've confirmed that you have the right dependencies installed, does your catalog look very different to mine? Is the data in the same folder?

[dwinski]

Thanks for trying, good to know it's working for you. My catalog looks exactly the same as yours but still getting the same two kedro ipython errors. After you did kedro new did you also do (this is what I did):
pip install -r src/requirements.txt
Could something else in the requirements.txt file have broken the environment?

[dwinski]

I'm going to try to recreate environment from scratch per your message above.

[dwinski]

I'm trying to recreate your steps from to create the spaceflights-dec15 environment but if I only do:
pip install kedro[pandas.csvdataset,pandasexceldataset]==0.17.6

then it appears I won't get IPython installed. So appears I do need to also do:
pip install -r src/requirements.txt

to get IPython and allow me to run >kedro ipython. Correct?

[dwinski]

Otherwise, how did you have IPython in this environment?

[dwinski]

Ok, after recreating environment per your steps I just ran:
pip install -r src/requirements.txt

... and now it's working! No idea what the problem was with the other environment but at least I can keep progressing on tutorial now. Thanks for your help. Will post again if I hit a wall with tutorial and can't solve it with what help is already available on interwebs.