Document how users can debug Kedro inside notebooks using the %load_node magic

[merelcht]

Description

Our debugging docs (https://kedro.readthedocs.io/en/stable/development/debugging.html) only explain how to do debugging in an IDE. Once we've added the line magic for making debugging nodes in notebooks easier, we should document the recommended flow of debugging within notebooks.

Context

Follow up on: #2009
More context: #1832

[stichbury]

Some further edits needed to the notebook pages (beyond what I've recently changed in #2031) as suggested by @yetudada:

In future, we might need to rejig this section so that it's easier to consume for beginner users. There is a lot of jargon that not even I understand (kernel, kernelspec, Kedro variables, etc) and I think we can greatly simplify the content.

It should answer the following:

What's our recommendation for when to use Jupyter notebooks?
What do we allow you to do in a Jupyter notebook?
Stuff with the catalog, running pipelines, seeing how many pipelines you have etc.
Visualising a pipeline
Editing catalog %reload_kedro
How do you open a Kedro-instance of a Jupyter notebook (kedro jupyter notebook or the stuff for the managed Jupyter notebook instance)

[AhdraMeraliQB]

Our debugging docs outline a debug flow that users can do from the CLI using hooks, with links to IDE-specific debug docs that live under their respective sections. Additionally, whilst this feature was designed to address the problem of debugging in notebooks, its definition is not restricted to only debugging.

As such, I would suggest that these line magic docs should live in https://docs.kedro.org/en/stable/notebooks_and_ipython/kedro_and_notebooks.html:

1. Add a section under https://docs.kedro.org/en/stable/notebooks_and_ipython/kedro_and_notebooks.html#reload-kedro-line-magic that describes the line magic and how to use it. Conclude section with a link to docs on using this for debugging. Make it very clear that this feature offers limited support.

2. Add a section to https://docs.kedro.org/en/stable/notebooks_and_ipython/kedro_and_notebooks.html#useful-to-know-for-advanced-users describing how to use the line magic for debugging. Make reference to limited support, linking users to the section added by Investigate if the ipython %debug line magic can be used for debugging Kedro in notebooks #2010 if working on an unsupported platform. Note that the line magic doesn't initiate a debugger, but can be used with one - perhaps incorporate this feature into this section instead of creating a new one?

On the primary debugging page, we can add a link to this section like we do for IDE debugging - perhaps something like

"For information on debugging in Jupyter notebooks, please visit "

CC @stichbury for opinions

[stichbury]

This all sounds good to me. That kedro_and_notebooks page still needs some work as described above and I think any revision therein should include those changes. Something you could pick up in a coming sprint, @AhdraMeraliQB ? Happy to pair with you!

[AhdraMeraliQB]

I think this is set to go in next week - CC @noklam, @merelcht for confirmation; Happy to pick it up and pair 😄

[noklam]

@stichbury @AhdraMeraliQB Yes, this should go next once the PR is merged. I'd also mention this again #2672, I think once we have the content in place, we need to organise in a way that it's easier to discover. I expected that debugging page link to our new docs instead of the debug hook.

[AhdraMeraliQB]

Closed by #3619