Visual graph element

[simongray]

Some kind of React graph element, presumably.

Should probably be a special element that can be opened or a separate page that is navigated to.

Requires moving the frontend dependencies from deps.edn into shadow-cljs.edn since I will need to use the NPM library integration of shadow-cljs.

[simongray]

https://mermaid-js.github.io/mermaid/#/./flowchart?id=flowcharts-basic-syntax

[simongray]

Having tried out vis.js in the form of react-graph-vis I didn't find it suitable for the following reasons:

1. The supported layouts didn't work well for the data we have.
2. There was seemingly no way to direct edge labels along the edges rather than on top of them.
3. There was seemingly no way to group nodes.
4. I found it relatively difficult to configure.

For this reason I have moved on to cytoscape.js packaged as /react-cytoscapejs. This solution fixes 1., 2., and 3. (using the "concentric" layout). It also feels easier to configure too, although I haven't fully configured it yet.

At this point I should probably also make a note about react-flow which on the surface seems like a well-documented, well-supported solution—and also something which could be used to edit graphs with at some point! However, I think react-flow is more suited to very small graphs with a clear "flow", rather than the type of graphs we must support, i.e. a single central node with a handful to several hundred outgoing connections.

[simongray]

While trying out Cytoscape.js I did found it to be an improvement over vis.js. However, it still has some major issues that at least require loading and configuring various extensions which may or may not solve these issues. The concentric layout is definitely preferable out of all of the layouts I have tried (including several custom ones), but I cannot configure the following:

• edge label backgrounds (they get smushed when appearing on top of other things)
• a minimum edge length (for small graphs, the edges are too short compared to the labels)
  • ... while there is seemingly no way to configure this, it may be possible to create a heuristic which adds minimumNodeDistance based on the amount of edges coming from the subject.
• expanding/collapsing compound node groups. I tried loading an extension for this, but couldn't get it to display anything different. I think the issue was that it required significant configuration.

Some illustrations:

Working well (cykel)

Too close (lexikalt begreb)

Too far (ord)

---

Based on these issue I have decided to try out a third method: D3's radial tidy tree layout. I am currently unsure how to best use D3 from Rum. Some reading:

• https://www.pluralsight.com/guides/using-d3.js-inside-a-react-app
• (unmaintained) https://github.com/react-d3-library/react-d3-library
• (reagent) https://github.com/gadfly361/rid3
• (inspiration) https://lambdaisland.com/blog/2018-04-26-d3-clojurescript

One thing that will not possible using a dendogram are self-referential relations. However, it should still be possible to include many-to-one relations as all of relations are from the same subject, so the objects can simply be duplicated many times over in the resulting tree.

[simongray]

Collapsible example: https://codepen.io/fernoftheandes/pen/kyQRNe

[simongray]

Having serious issues implementing the D3 radial tree for several reasons:

• The collapsible example is using an old version of D3 which I don't want to use, so it needs to be rewritten
• The observable version is hard to get clean source code for (or my JS foo is not enough)
• The amount of JS to translate to CLJS is quite tiring. I tried mixing with pure JS code, but ran into the issues described in the bullets above.

---

Now I just discovered that there is actually a radial tree layout in vega too, which seems to look very similar—perhaps even better—and it seems much more compatible with my needs. I will go investigate.

• ClojureScript has two mature packages for Vega viz: Hanami and Oz
• There is also the officially supported react-vega whose use from CLJS has been documented in a blog post.

I think I will attempt the React interop to begin with.

[simongray]

Vega attempt being tracked in: https://github.com/kuhumcst/DanNet/tree/feature/31d-vega

[simongray]

More inspiration: https://vega.github.io/vega/examples/tree-layout/

[simongray]

https://openbase.com/categories/js/best-react-graph-libraries

[simongray]

This looks interesting: https://unovis.dev/gallery

Perhaps an alternative to a graph could be the expanded sankey?

[simongray]

Some very interesting force graphs here: https://github.com/vasturiano/react-force-graph

I might also take a look at this one again: https://www.npmjs.com/package/react-d3-tree

[simongray]

Since the added JS dependencies take up quite a lot of additional MBs, I will likely need to use a shadow-cljs module of some kind.

[simongray]

This also looks promising, especially if the performance is good: https://github.com/reaviz/reagraph

I tried getting react-force-graph to work, but it's having some dependency issue in shadow-cljs that I'm unsure how to solve.

[simongray]

Thomas Heller gave me this input for the issue I'm having with react-force-graph.

Gonna reagraph first and then attempt the above as a plan B.

[simongray]

Current status:

With regard to reagraph, I simply can't figure out why it isn't working. The canvas shows up, but actually displaying nodes and edges isn't happening.

As react-force-graph, the solution provided by theller did in fact mute the warning, but the module compilation error is still present and in fact crashes Firefox after a short while.

[simongray]

I managed to get react-force-graph working by updating some deps (shadow-cljs, Rum, reagent-utils, and React itself). The basic layout is working, but adding labels even seems to be quite a bit of work, unfortunately.

Replicating the old-school radial layouts that Bolette loves might be possible with https://uber.github.io/react-vis/examples/showcases/sunbursts

[simongray]

To create proper nodes with text I'll have to manually style them with nodeCanvasObject, e.g. see example here: https://github.com/vasturiano/react-force-graph/blob/master/example/custom-node-shape/index-canvas.html

[simongray]

Word cloud visual concept

One thing I'm playing with is displaying collections of synsets as word clouds, in order to have something meaningful to show in every case, e.g. rather than linking to each synset individually, a word cloud stands in as the single object that is linked to for each particular relation. This will probably scale quite well, since more important words will be more prominent than less important ones in cases where there are many connections.

Obviously, this requires the importance of a synset to be predetermined and pre-fetched as part of the query, e.g. it could be the synset in-degree stored as an attribute calculated after creating the database. I'm not sure how expensive these calculations are, though.

react-force-graph (now working) should allow me to put a custom word cloud as the object for collections/sets. In fact, it might actually be possible to embed this lib itself, since it can make alternate size nodes.

[simongray]

Ok, so naively querying for all synsets is OK, but querying for incoming synset connections to synsets is very costly, at least in a single query.

Perhaps it is less expensive to query first for all synsets and then do the incoming connections bit sequentially?

[simongray]

I turns out that querying for all incoming connections to synsets is doable in ~6 minutes, but querying for only those coming from synset is... hard enough that I haven't concluded how long it takes.

It seem that the solution is to use only the unfiltered connections to calculate the indegree, i.e. the 6 minute query.

[simongray]

[simongray]

It turns out that the indegree is also a great way to display a small summary for the long lists of synsets that sometime appear for different synset pages, e.g. http://localhost:3456/dannet/data/synset-36945

I will need to design the summary/details element currently in use, though, as that no longer is for for purpose.

[simongray]

The simple word cloud in https://npm.io/package/react-wordcloud looks quite attractive and I can make callbacks to change the page when clicking a word.

[simongray]

I got a word cloud going with d3-cloud, however this brings also brings in D3 and associated deps, so the main.js size has gone up from 1.13 MB to 1.65 MB, i.e. roughly half a MB. This sucks, but it's hard to see how I can avoid it.

[simongray]

Examples:

http://localhost:3456/dannet/external/en/oewn-00244785-n (has_domain_topic)

http://localhost:3456/dannet/data/synset-16749 (hyponym)

[simongray]

Another thing I have not considered is accessibility. I guess no one would care to randomly jump through a thousand randomly ordered clickable eleemnets, so perhaps it's ok to leave out. They are available in the list anyway, after all.

[simongray]

I think I will first attempt to integrate the word cloud into the basic entity page by creating a stylised checkbox widget for toggling word cloud/list views.

[simongray]

I am recently satisfied with the word cloud + display options select dropdown in 72a1860.

Next on the visualisation agenda is combining this word cloud stuff into a network graph.

[simongray]

I've managed to get an example workingbased on https://observablehq.com/@d3/radial-tree/2

The original plan was to combine it with the word clouds, however, now I think it might be more beneficial to simply duplicate the old Yahoo YUI widget on Andre Ord:

I would be placing a visualisation widget at the top of the Semantic Relations section and use only those relations for the visualisation. One of these visualisations would then be a D3 copy of the Andre Ord viz.

[simongray]

Closing as of 8862dfe since the MVP for the Andre Ord style radial diagram is done and the word cloud functionality itself is too.

I will make several smaller issues for the remaining stuff that needs to be done, e.g. fixes for and integrated functionality of these visualisations + the (now) nice-to-have visualisation combining word clouds and the radial diagram.