Improve get started docs and guide to working with notebooks #2031

stichbury · 2022-11-16T16:17:37Z

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Ready for review by @yetudada @datajoely @merelcht and anyone else interested!

Sorry, sorry, sorry, it looks to be a massive PR 🙀

It's not as bad as it looks but I've renamed some pages and changes some sub-section names so I've had to modify a few files extra just to fix broken links.

I suggest you take a look mostly at the resulting output which I've put here:
https://stichbury.github.io/

There are some conflicts now as a result of other PR submissions but I'll handle that in parallel with any review.

Changelog

Revised the Introduction to make it short and sweet.
Revised the Get Started section. Gone is "Hello Kedro". Gone are the installation pre-requisites (that's just part of the Install Kedro page now). Gone is the "Standalone use of the data catalog - woot woot" and GONE is the page on Kedro starters.
I've reordered the create project material to put the project structure breakdown in the section that introduces key concepts and shorten the Iris tutorial to the bare minimum. I did add visualisation at this point though, to highlight Kedro Viz, as I felt it was coming far too late in the spaceflights tutorial and needed to be more prominent as a feature.
Added a TL;DR page to Get Started which some people could probably just use as-is and ignore the rest of the section.
Starters material has moved into a new section all about "Kedro project setup". Much of that section still needs review/revision but I have updated the Starters page so it reads more clearly.

I think that's about all, apart from some minor fixes (Github -> GitHub) and fixes to links for Slack.

Checklist

Read the contributing guidelines
Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added a description of this change in the RELEASE.md file
Added tests to cover my changes

Mostly removing text and simplifying FTW Signed-off-by: Jo Stichbury <jo.stichbury@quantumblack.com>

Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>

….com/kedro-org/kedro into kedro-1874-spaceflights-improvements

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

merelcht

Just some minor comments from me. This is a great improvement! The shorter pages make the docs much easier to follow 👍 ⭐

docs/source/get_started/summary.md

jmholzer

Amazing work! These docs are so well written, clear and easy to follow 🚀🚀.

I especially appreciate your work on notebooks.

I have left a few comments, none too major.

docs/source/faq/architecture_overview.md

docs/source/get_started/install.md

docs/source/get_started/summary.md

docs/source/notebooks_and_ipython/kedro_and_notebooks.md

yetudada

I love these changes! I left minor comments.

docs/source/contribution/contribute_to_kedro.md

docs/source/faq/faq.md

docs/source/get_started/install.md

README.md

docs/source/contribution/technical_steering_committee.md

docs/source/faq/faq.md

docs/source/get_started/new_project.md

docs/source/notebooks_and_ipython/kedro_and_notebooks.md

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

…dro-org/kedro into 1989-improve-get-started-docs

stichbury · 2022-11-23T16:00:37Z

All updated following reviews by @noklam @merelcht @yetudada and @jmholzer

Thanks for your feedback. I've made loads of changes -- we're now at 97 commits for this PR 😆

Please can I get a final approval to merge?

The latest and greatest is visible on https://stichbury.github.io/

noklam

Approved ⭐️ x100

jmholzer

Thanks so much for this! 🎉

* Update attrs requirement from ~=21.3 to >=22.1.0,<23.0 in /dependency Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Updated the documentation to reflect the code changes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Add Comment Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Changed Attrs pin to attrs>=20.0, <23.0 Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Updated Release Notes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Updated Lower Bound to be 21.3 Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Updated Release Notes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * Update RELEASE.md Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Remove upper bound * Add a video dataset (#1312) This commit adds a video dataset that can read and write video files. The dataset is backed by OpenCVs video reader and writer and different buffer protocols such as PIL.Image and numpy.ndarray. There is one class for iterables (e.g. list of frames) and one for generators. Since large videos might not fit in memory, this ability allows us to read one frame from the video file, transform the frame and write it to another video file. Thus there is only need for one frame at a time in memory. The different codecs that are supported depends on the codecs installed and how OpenCV was compiled. Signed-off-by: Daniel Falk <daniel.falk.1@fixedit.ai> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Update fsspec requirement in /dependency (#2024) Updates the requirements on [fsspec](https://github.com/fsspec/filesystem_spec) to permit the latest version. - [Release notes](https://github.com/fsspec/filesystem_spec/releases) - [Commits](fsspec/filesystem_spec@2021.04.0...2022.11.0) --- updated-dependencies: - dependency-name: fsspec dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Update pip-tools requirement from ~=6.9 to ~=6.10 in /dependency (#2023) Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version. - [Release notes](https://github.com/jazzband/pip-tools/releases) - [Changelog](https://github.com/jazzband/pip-tools/blob/master/CHANGELOG.md) - [Commits](jazzband/pip-tools@6.9.0...6.10.0) --- updated-dependencies: - dependency-name: pip-tools dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Add execution_options to SQLQueryDataSet (#1865) * Add execution_options to SQLQueryDataSet Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * Update RELEASE.md Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * Fix lint issues Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * Add a check in the unit test Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * Add test for connection reuse behaviour Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * Inject execution_options at load time Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> * enhance doc and fix incorrect example Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * More docs change Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Fixing typos Signed-off-by: Nok <nok_lam_chan@mckinsey.com> Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> Co-authored-by: Nok <nok_lam_chan@mckinsey.com> Co-authored-by: Nok Lam Chan <mediumnok@gmail.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Update attrs requirement from ~=21.3 to >=22.1.0,<23.0 in /dependency Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Updated the documentation to reflect the code changes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Add Comment Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Changed Attrs pin to attrs>=20.0, <23.0 Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Updated Release Notes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Updated Lower Bound to be 21.3 Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Updated Release Notes Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Update links to Slack & add note about security (#2033) Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Apply OpenSSF Best Practices Badge (#2034) Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Add SVMLight DataSet (#1992) * Add SVMLight DataSet Add DataSet for svmlight/libsvm files using scikit-learn library as backend. Resolves #1972 Signed-off-by: Kirill Korotkov <korotkovkm@gmail.com> * Pin scikit-learn version to work with python 3.7 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Update dataset docstring Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Pin requirements to work with python 3.7 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add requirements to setup.py Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add changes to dataset docstring Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Dummy commit to retrigger CI pt1 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Dummy commit to retrigger CI pt2 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Update dataset dockstring Signed-off-by: Kirill Korotkov <korotkovkm@gmail.com> Signed-off-by: Kirill Korotkov <korotkovkm@gmail.com> Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Co-authored-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Update RELEASE.md Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Improve get started docs and guide to working with notebooks (#2031) * Revised the Introduction to make it short and sweet. * Revised the Get Started section. Gone is "Hello Kedro". Gone are the installation pre-requisites (that's just part of the Install Kedro page now). Gone is the "Standalone use of the data catalog - woot woot" and GONE is the page on Kedro starters. * Reordered the create project material to put the project structure breakdown in the section that introduces key concepts and shorten the Iris tutorial to the bare minimum. I did add visualisation at this point though, to highlight Kedro Viz, as I felt it was coming far too late in the spaceflights tutorial and needed to be more prominent as a feature. * Added a TL;DR page to Get Started which some people could probably just use as-is and ignore the rest of the section. * Starters material has moved into a new section all about "Kedro project setup". Much of that section still needs review/revision but I have updated the Starters page so it reads more clearly. * Improved the Kedro-Viz page somewhat (still more to come for Plotly) * Notebooks/IPython materials now merged and simplified Signed-off-by: Nok <nok_lam_chan@mckinsey.com> * Remove upper bound Signed-off-by: Nok <nok_lam_chan@mckinsey.com> Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Daniel Falk <daniel.falk.1@fixedit.ai> Signed-off-by: Nok <nok_lam_chan@mckinsey.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Clotilde Guinard <clotilde_guinard@hotmail.fr> Signed-off-by: Kirill Korotkov <korotkovkm@gmail.com> Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Co-authored-by: Nok Lam Chan <mediumnok@gmail.com> Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Co-authored-by: Daniel Falk <daniel@da-robotteknik.se> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: clotildeguinard <clotilde_guinard@hotmail.fr> Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com> Co-authored-by: Cyril Korotkov <korotkovkm@gmail.com> Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Co-authored-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

stichbury and others added 30 commits October 20, 2022 11:36

Update first two pages of the tutorial

c9562ec

Mostly removing text and simplifying FTW Signed-off-by: Jo Stichbury <jo.stichbury@quantumblack.com>

Update tutorial docs

66f770f

Tutorial improvements

45537b0

Update create pipelines docs

36e1e90

Further spaceflights enhancements

df7a44a

Merge branch 'main' into kedro-1874-spaceflights-improvements

0753c7c

Updates

6346146

More changes to docs structure and content

05c8faa

Merge branch 'main' into kedro-1874-spaceflights-improvements

bf40827

Few more tweaks

e8bf8f1

Update kedro-viz_visualisation.md

72063a0

Update build-docs.sh

4813607

Update .gitignore to add docs/temp directory

da279d4

Update docs/source/tutorial/create_pipelines.md

a546b13

Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>

Merge branch 'kedro-1874-spaceflights-improvements' of https://github…

a4d40a1

….com/kedro-org/kedro into kedro-1874-spaceflights-improvements

Merge branch 'main' into kedro-1874-spaceflights-improvements

74f0e9a

More changes to add modular pipelines

c50f32b

Merge branch 'kedro-1874-spaceflights-improvements' of https://github…

4d52da5

….com/kedro-org/kedro into kedro-1874-spaceflights-improvements

Few more updates

7ae6371

Pre-commit checks

7a24d8d

Merge branch 'main' into kedro-1874-spaceflights-improvements

f517323

Fix a few issues

2ed06f0

Merge branch 'kedro-1874-spaceflights-improvements' of https://github…

0b482a0

….com/kedro-org/kedro into kedro-1874-spaceflights-improvements

Update docs/source/tutorial/set_up_data.md

5dd4c68

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

Update docs/source/tutorial/set_up_data.md

1048b6e

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

Update docs/source/tutorial/spaceflights_tutorial.md

49138e2

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

Updates to get started section

233f41e

Merge branch 'main' into kedro-1874-spaceflights-improvements

7a34ec0

Merge changes by picklejuicedev

c09cbbb

Update add_another_pipeline.md

887553e

Update glossary.md

550ac04

merelcht approved these changes Nov 21, 2022

View reviewed changes

docs/source/get_started/summary.md Show resolved Hide resolved

docs/source/get_started/summary.md Show resolved Hide resolved

Update summary.md

d6cf872

jmholzer reviewed Nov 21, 2022

View reviewed changes

yetudada reviewed Nov 22, 2022

View reviewed changes

stichbury and others added 13 commits November 23, 2022 10:53

Update docs/source/faq/faq.md

11562de

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/get_started/install.md

a75f2a0

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update README.md

4c064bb

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/faq/faq.md

6798ef2

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/get_started/new_project.md

8ed2088

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/get_started/summary.md

ba8d8ed

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/get_started/summary.md

4e7d967

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/index.rst

d42608f

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/visualisation/kedro-viz_visualisation.md

f7c0c4d

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/kedro_project_setup/starters.md

5601321

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/kedro_project_setup/starters.md

9c3d637

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

Update docs/source/get_started/install.md

aa9475a

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

Update docs/source/get_started/summary.md

1d60969

Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>

stichbury mentioned this pull request Nov 23, 2022

Document how users can debug Kedro inside notebooks using the %load_node magic #2011

Closed

stichbury added 2 commits November 23, 2022 15:55

Update following review

2cfbf2a

Merge branch '1989-improve-get-started-docs' of https://github.com/ke…

d6d57db

…dro-org/kedro into 1989-improve-get-started-docs

stichbury requested review from yetudada, noklam and jmholzer November 23, 2022 15:58

noklam approved these changes Nov 23, 2022

View reviewed changes

jmholzer approved these changes Nov 23, 2022

View reviewed changes

stichbury merged commit ebf3d64 into main Nov 23, 2022

stichbury deleted the 1989-improve-get-started-docs branch November 23, 2022 21:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve get started docs and guide to working with notebooks #2031

Improve get started docs and guide to working with notebooks #2031

stichbury commented Nov 16, 2022 •

edited by gitpod-io bot

Loading

merelcht left a comment

jmholzer left a comment

yetudada left a comment

stichbury commented Nov 23, 2022

noklam left a comment

jmholzer left a comment

Improve get started docs and guide to working with notebooks #2031

Improve get started docs and guide to working with notebooks #2031

Conversation

stichbury commented Nov 16, 2022 • edited by gitpod-io bot Loading

Description

Changelog

Checklist

merelcht left a comment

Choose a reason for hiding this comment

jmholzer left a comment

Choose a reason for hiding this comment

yetudada left a comment

Choose a reason for hiding this comment

stichbury commented Nov 23, 2022

noklam left a comment

Choose a reason for hiding this comment

jmholzer left a comment

Choose a reason for hiding this comment

stichbury commented Nov 16, 2022 •

edited by gitpod-io bot

Loading