Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation on how to visualise Matplotlib plots on Kedro-Viz #1820

Merged
merged 40 commits into from
Sep 9, 2022
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
81ab6b0
Vhange demo gif to include plots
Aug 26, 2022
14d7849
Add sction about plots in experiment tracking view
Aug 26, 2022
c3dd034
Add gif of plot comparison
Aug 30, 2022
deb0f67
Replace run list screenshot to include plots
Aug 30, 2022
84d4b04
Replace large demo to include plots
Aug 30, 2022
613086c
Merge branch 'main' into docs/plots-on-exp-tracking
AhdraMeraliQB Aug 30, 2022
46bf8d4
Separate existing docs into two sections under common subheading
Aug 30, 2022
3ed0cf0
Add description to Matplotlib section
Aug 30, 2022
10118ad
Merge branch 'main' into docs/add-matplotlib-plots
AhdraMeraliQB Aug 30, 2022
c91b24a
Add Matplotlib node example
Aug 30, 2022
ef31650
Merge branch 'docs/add-matplotlib-plots' of github.com:kedro-org/kedr…
Aug 30, 2022
76b15c9
Change image name
Aug 31, 2022
b68225d
Add screenshots to example
Aug 31, 2022
f434524
Lint changes
Aug 31, 2022
5fe23a8
Update images to include all charts
Aug 31, 2022
dfd66e0
Merge branch 'main' into docs/plots-on-exp-tracking
antonymilne Sep 2, 2022
be548f5
Merge branch 'docs/plots-on-exp-tracking' into docs/add-matplotlib-plots
antonymilne Sep 2, 2022
8864f28
Update docs/source/tutorial/set_up_experiment_tracking.md
AhdraMeraliQB Sep 5, 2022
8630a19
Update docs/source/tutorial/set_up_experiment_tracking.md
AhdraMeraliQB Sep 5, 2022
d77f4cb
Add suggestions from review
Sep 5, 2022
fd77271
Update docs/source/tutorial/set_up_experiment_tracking.md
AhdraMeraliQB Sep 5, 2022
b7595af
Merge branch 'main' into docs/plots-on-exp-tracking
AhdraMeraliQB Sep 5, 2022
259dd81
Change heading tense
AhdraMeraliQB Sep 5, 2022
943e8f3
Change api link to relative
AhdraMeraliQB Sep 5, 2022
6d9a299
Change inline code to code block to follow convention
AhdraMeraliQB Sep 5, 2022
d66e621
Change to Sphinx syntax
AhdraMeraliQB Sep 5, 2022
2cee8eb
Remove empty lines
AhdraMeraliQB Sep 5, 2022
86e93af
Edit paragraph
AhdraMeraliQB Sep 5, 2022
638e793
Replace randomised node
Sep 5, 2022
2ce2e1f
Fix typo and move note
Sep 5, 2022
7433085
Fix linting
Sep 5, 2022
7b1afe2
Merge branch 'docs/plots-on-exp-tracking' into docs/add-matplotlib-plots
AhdraMeraliQB Sep 5, 2022
dfe47f7
Merge branch 'main' into docs/add-matplotlib-plots
AhdraMeraliQB Sep 6, 2022
eca1708
Change Kedro-Viz plot visualisations description
Sep 8, 2022
52fc284
Merge branch 'main' into docs/add-matplotlib-plots
AhdraMeraliQB Sep 8, 2022
ca5b7c7
Update docs/source/tutorial/visualise_pipeline.md
AhdraMeraliQB Sep 8, 2022
8da3c25
Add explanation for versioned flag
Sep 8, 2022
14d7b98
Merge branch 'docs/add-matplotlib-plots' of github.com:kedro-org/kedr…
Sep 8, 2022
2911111
Merge branch 'main' into docs/add-matplotlib-plots
AhdraMeraliQB Sep 9, 2022
b2b04e3
Merge branch 'main' into docs/add-matplotlib-plots
AhdraMeraliQB Sep 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# Upcoming Release 0.18.3

## Major features and improvements
* Implemented autodiscovery of project pipelines.
* The line magic `%reload_kedro` now accepts keywords arguments, e.g. `%reload_kedro --env=prod`.

## Bug fixes and other changes
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/source/tutorial/set_up_experiment_tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ You can now access, compare and pin your runs by toggling the `Compare runs` but

## View and compare plot data

Experiment tracking in Kedro-Viz also supports the display and comparison of plots, such as Plotly and Matplotlib. Once you have [created your plots](../tutorial/visualise_pipeline.md#visualise-plotly-charts-in-kedro-viz) in your kedro-project, you must set the versioned flag to `true` within the project catalog to include them in experiment tracking.
Experiment tracking in Kedro-Viz also supports the display and comparison of plots, such as Plotly and Matplotlib. Once you have [created your plots](../tutorial/visualise_pipeline.md#visualise-charts-in-kedro-viz) in your kedro-project, you must set the versioned flag to `true` within the project catalog to include them in experiment tracking.

```
# conf/base/catalog.yml
Expand Down
79 changes: 73 additions & 6 deletions docs/source/tutorial/visualise_pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,13 @@ kedro viz --load-file my_shareable_pipeline.json
This will visualise the pipeline visualisation saved as `my_shareable_pipeline.json`.


## Visualise Plotly charts in Kedro-Viz
## Visualise charts in Kedro-Viz

[Plotly](https://plotly.com/python/) is a free and open source Python library that allows you to make interactive, publication-quality graphs. With the Plotly integration on Kedro-Viz, you can output your interactive charts as part of your pipeline visualisation.
Kedro-Viz supports [Plotly](https://plotly.com/python/) and [Matplotlib](https://matplotlib.org/) integrations so that you can share your data insights while exploring your pipeline.

### Visualise Plotly charts in Kedro-Viz

Kedro-Viz aims to help users communicate different aspects of their workflow through an interactive flowchart. With the Plotly integration, we take one step further in this direction to allow our users to share their data insights effectively while exploring the pipeline.
[Plotly](https://plotly.com/python/) is a free and open source Python library that allows you to make interactive, publication-quality graphs. With the Plotly integration on Kedro-Viz, you can output your interactive charts as part of your pipeline visualisation.

We have also used the Plotly integration to allow users to [visualise metrics from experiments](../logging/experiment_tracking.md).

Expand All @@ -115,7 +117,7 @@ You can view Plotly charts in Kedro-Viz when you use Kedro's plotly datasets.

There are two types of Plotly datasets in Kedro: the `plotly.PlotlyDataSet` and `plotly.JSONDataSet`.

### [`plotly.PlotlyDataSet`](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.plotly.PlotlyDataSet.html#kedro.extras.datasets.plotly.PlotlyDataSet)
#### [`plotly.PlotlyDataSet`](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.plotly.PlotlyDataSet.html#kedro.extras.datasets.plotly.PlotlyDataSet)

To use this dataset, configure your plot in the `catalog.yml` file. This dataset only supports [Plotly Express](https://plotly.com/python/plotly-express).

Expand Down Expand Up @@ -163,7 +165,7 @@ shuttle_passenger_capacity_plot:
```


### [plotly.JSONDataSet](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.plotly.JSONDataSet.html#kedro.extras.datasets.plotly.JSONDataSet)
#### [`plotly.JSONDataSet`](https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.plotly.JSONDataSet.html#kedro.extras.datasets.plotly.JSONDataSet)

To use this dataset, configure your plot in your Kedro node. This dataset supports [Plotly Express](https://plotly.com/python/plotly-express) and [Plotly Graph Objects](https://plotly.com/python/graph-objects/).

Expand Down Expand Up @@ -223,7 +225,7 @@ shuttle_passenger_capacity_plot:



Once the above setup is completed, you can do a `kedro run` followed by `kedro viz` and your Kedro-Viz pipeline will show a new dataset type with icon ![](../meta/images/plotly-icon.png) . Click on the node to see a small preview of your Plotly chart in the metadata panel.
Once the above setup is completed, you can do a `kedro run` followed by `kedro viz` and your Kedro-Viz pipeline will show a new dataset type with icon ![](../meta/images/chart-icon.png) . Click on the node to see a small preview of your Plotly chart in the metadata panel.

![](../meta/images/pipeline_visualisation_plotly.png)

Expand All @@ -233,3 +235,68 @@ You can view the larger visualisation of the chart by clicking the 'Expand Plotl


![](../meta/images/pipeline_visualisation_plotly_expand.png)


### Visualise Matplotlib charts in Kedro-Viz

[Matplotlib](https://matplotlib.org/) is a Python library for creating static, animated, and interactive visualisations. Integrating Matplotlib into Kedro-Viz allows you to output your charts as part of your pipeline visualisation.

```{note}
The MatplotlibWriter dataset converts Matplotlib objects to image files. This means that Matplotlib charts within Kedro-Viz are static and not interactive, unlike the Plotly charts seen above.
```

You can view Matplotlib charts in Kedro-Viz when you use the [Kedro MatplotLibWriter dataset](/kedro.extras.datasets.matplotlib.MatplotlibWriter). You must update the `src/requirements.txt` file in your Kedro project by adding the following dataset to enable Matplotlib for your project:

```
kedro[matplotlib.MatplotlibWriter]==0.18.2
```

To use this dataset, configure your plot in your Kedro node. The below functions can be added to the `nodes.py` and `pipeline.py` files respectively.
AhdraMeraliQB marked this conversation as resolved.
Show resolved Hide resolved

```python
# nodes.py
import matplotlib.pyplot as plt


def create_confusion_matrix(companies: pd.DataFrame):
AhdraMeraliQB marked this conversation as resolved.
Show resolved Hide resolved
actuals = [0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1]
predicted = [1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1]
data = {"y_Actual": actuals, "y_Predicted": predicted}
df = pd.DataFrame(data, columns=["y_Actual", "y_Predicted"])
confusion_matrix = pd.crosstab(
df["y_Actual"], df["y_Predicted"], rownames=["Actual"], colnames=["Predicted"]
)
sn.heatmap(confusion_matrix, annot=True)
return plt


# pipeline.py
def create_pipeline(**kwargs) -> Pipeline:
"""This is a simple pipeline which generates a plot"""
return pipeline(
[
node(
func=create_confusion_matrix,
inputs="companies",
outputs="dummy_confusion_matrix",
),
]
)
```

You must also specify the output type in the `catalog.yml` file, like below.

```yaml
reporting.dummy_confusion_matrix:
type: matplotlib.MatplotlibWriter
filepath: ${base_location}/08_reporting/dummy_confusion_matrix.png
versioned: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be useful to add a sentence about why versioned: true has been set here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Added an explanation in 14d7b98.

```

Once this setup is completed, you can do a `kedro run` followed by `kedro viz` and your Kedro-Viz pipeline will show a new dataset node with this icon ![](../meta/images/chart-icon.png). Click on the node to see a small preview of your Matplotlib image in the metadata panel.

![](../meta/images/pipeline_visualisation_matplotlib.png)

You can view the larger visualisation of the chart by clicking the 'Expand Matplotlib Image' button on the bottom of the metadata panel.

![](../meta/images/pipeline_visualisation_matplotlib_expand.png)