Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spike: Try to reproduce non-deterministic rendering #2057

Closed
1 task done
astrojuanlu opened this issue Aug 23, 2024 · 5 comments · Fixed by #2185
Closed
1 task done

Spike: Try to reproduce non-deterministic rendering #2057

astrojuanlu opened this issue Aug 23, 2024 · 5 comments · Fixed by #2185

Comments

@astrojuanlu
Copy link
Member

Description

I'm always frustrated when I do kedro viz run twice and each time the rendered DAG is different.

Context

I was reviewing #1966 and I wanted to see if the rendered pipeline is the same, but it's difficult because the sorting of the nodes can differ.

Checklist

  • Include labels so that we can categorise your feature request
@ravi-kumar-pilla
Copy link
Contributor

This will also enable having autoreload of Kedro-Viz in VSCode extension.

@astrojuanlu
Copy link
Member Author

A bit of history: as far as I understand, this is where the algorithm was introduced #185

@astrojuanlu
Copy link
Member Author

And I don't see any references to randomness in the original paper that defines the Cassowary algorithm https://doi.org/10.1145/504704.504705 and no mention in the kiwi.js repository either https://github.com/IjzerenHein/kiwi.js/ so my guess is that the randomness actually comes from Kedro Viz? 🤔

@astrojuanlu astrojuanlu changed the title Deterministic rendering Spike: Try to reproduce non-deterministic rendering Nov 7, 2024
@noklam
Copy link
Contributor

noklam commented Nov 11, 2024

Is there an issue with reproducing this? I thought this is always the case. I just do kedro viz on an example project and it resulted in different layout:

image
image

@rashidakanchwala
Copy link
Contributor

rashidakanchwala commented Nov 12, 2024

And I don't see any references to randomness in the original paper that defines the Cassowary algorithm https://doi.org/10.1145/504704.504705 and no mention in the kiwi.js repository either https://github.com/IjzerenHein/kiwi.js/ so my guess is that the randomness actually comes from Kedro Viz? 🤔

Thanks, @astrojuanlu, for your help with the above. This clarified that the issue was with Kedro-Viz. In the graph, we pass nodes, edges, and layers. I noticed that while the ordering of nodes and layers remained consistent, the edges’ order varied with each time the graph was calculated. I’ve added a sortEdges function to reduce randomness by ensuring we pass the same edges dictionary each time. It seems to be working now.

I’ll open a PR so others can test it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants