[Experiment] Distinctive MemoryDataset view on flowchart #1706
Conversation
rashidakanchwala
requested review from
astrojuanlu,
datajoely,
yetudada,
merelcht,
NeroOkwa and
noklam
and removed request for
tynandebold and
yetudada
|
It's beautiful! I'd suggest this should be default behaviour already - but it's should be really quick to get some user validation of that |
|
I think this is very nice functionality, speaking as someone who has a loot of memorydatasets in my pipeline. In my case it's not relevant, but I know some users use "dummy" inputs/outputs to force node order execution, I'm not sure if they tend to use memorydatasets or another type. Maybe a user-configurable toggle per dataset type to fade/not fade? |
| @@ -171,6 +172,10 @@ export const drawNodes = function (changed) { | |||
| ) | |||
| .classed('pipeline-node--data', (node) => node.type === 'data') | |||
| .classed('pipeline-node--task', (node) => node.type === 'task') | |||
| .classed( | |||
| 'pipeline-node--memory-data', | |||
| (node) => flags?.diffMemoryDatasets && node?.dataset?.includes('memory') | |||
There was a problem hiding this comment.
This looks good. Can we have a DatasetType check instead of string matching ?
There was a problem hiding this comment.
yes, i think we will eventually refactor and do it properly. for now this is just a quick dirty experiment
|
This looks great already to distinguish MemoryDataset. May be we can extend this to opt-in color-code dataset feature (we color datasets based on type - for easier debugging). Thank you ! |
|
I really like this idea (more than #1707). As I said there, I'm wondering if eventually we could provide a more generic customisation feature that would allow users to have more control over this, instead of adding a toggle. |
|
@astrojuanlu -- I agree. After speaking to a few people, I do realise that while some people might like this, others might not find it so usable. They might want to differentiate other datasets instead. Perhaps we need to build some sort of customisability, as you mentioned in the other PR, to allow users to distinguish datasets rather than us doing it for them. Ravi also mentioned color-coding datasets, and as we have too many datasets, giving each a color-code might end up being more confusing and less visually appealing? well, this is subjective, isnt it? hence, providing them some sort of customisability where they can decide how they want to highlight or distinguish certain datasets over others would be better. I will try and get more feedback on this short experiment. But keep the comments coming :) |
|
Closing this experiment PR for now -- allowing users to differentiating datasets on kedro-viz seems like a good idea. We are going to try and do #1148 first and see how that picks up with users. |
Description
Suggested for quick experimentation by @datajoely: MemoryDatasets are non-persistent datasets, meaning they are not saved after a run. Some flowcharts contain numerous MemoryDatasets, and Kedro-viz currently lacks a way to differentiate them from persistent datasets. This PR introduces opacity to MemoryDatasets, making them easily distinguishable on the flowchart. The transparency also signifies their non-persistent nature.
Currently this feature is under an experiment flag.
Development notes
QA notes
Checklist
RELEASE.mdfile