Feature: Simple Filtering

[benloh]

Joshua's writeup of simple filtering (from https://indiana-my.sharepoint.com/:w:/g/personal/jdanish_iu_edu/EWSbxa56u9dOtbKT3u3fQhEBMYidfLTC1c4cEfr6hSSRlw?rtime=d4osTmMR2Eg):

Templates

1. Add a field to all fields in the template for type and for now only include number or text

2. If we have time, add validation for number, but if not I can add it if the property is there

Actual filter

1. Create a small movable pop up window that triggers from the tab bar for filtering

   1. This is tabbed with 2 tabs to start: Nodes and Edges

   2. Help text drawn from this template

2. In each tab:

   1. By type: list node / edge types with check boxes to include / exclude them. All included by default.

   2. Then, this has a list of node and edge fields, and each one has a drop down as follows:

      a. If the type is “text” then it includes “not filtered”, “contains” or “doesn’t contain”
      1. If you select contains or doesn’t contain, you can type a string and it will filter accordingly
      2. Ideally we have a way of using multiple strings with quotes to do multi-filter, but if not the basics are a great start

      b. If the type is “number” then it includes some basic mathematical comparisons such as =, !=, >, < and a space for a number
      1. (Note, auto-calculated degrees count as a number)
      2. If we can add calculated fields like “nearest neighbor” then do so. (maybe specify the key 1 or 2)

3. In the bottom of the popup

   1. A clear and close button

   2. Clear clears all the filters / re-sets it

   3. I assume we don’t need apply as it will happen automatically with each selection

Features

• Filter status display line
• Date filtering

Future Features

• Highlight -- Fade filtered objects instead if hiding them.

[benloh]

@jdanish @kalanicraig So...filtering is anything but simple. While a basic data filtering algorithm is fairly easy, multi-field filtering and a UI that is dynamically generated from the template is fairly complex. Two questions about the filtering design:

1. Given how much room the filter takes up, I figured it was easier putting it in the info panel/tab bar along with everything else. That way users won't have to manage an extra window. Is that OK?

Here's a rough first pass. It can be tightened up.

2. When you set a filter are you saying "Search for and show me nodes that match xyz"? or are you saying "Search for and hide nodes that match xyz"?

I was finding myself somewhat confused by it as I was playing with it. At first I expected it to behave like the "Search" function. Searching for a label that "contains" C means show me all the labels that have C.

But I imagine whether it's show or hide depends on the nature of the filtering you're wanting to do.

Do you have any examples of the types of filtering you want to do / have done on existing graphs? Should it be show or hide? Or does that actually need to be a separate command option for the whole filter? e.g. at the top we might have a drop down that says "Show all nodes that match" vs "Hide all nodes that match"?

[kalanicraig]

Simple stuff first: I love the tab option, but Joshua points out that we’d need a filter warning somewhere in the always-visible UI area that alerts users to the presence of a filter (e.g. “The visualization and node/edge tables are currently filtered. Click the “Filters” tab to edit or clear the filters and display the whole graph’). Because there’s a “does not contain” option in the search, we should only be working from a “show” not a “show/hide” assumption. Most of the network software packages out there do a contain/does-not-contain option rather than a show/hide-matches anyway. In the short-to-medium term, the network-theory use of filtering we’d likely use most is to show ego networks (subnetwork context for a single node that shows its edges and connected nodes, regardless of whether the connected nodes/edges meet the filtering conditions). That’s usually done with a “depth” filter that’s added to the main node/edge-type selection or the text-contains keyword. A single-node-search example: If we had typed in “Agrippina” as our label-contains keyword, and selected a depth of 1, then all nodes regardless of node label or type with a direct connection to Agrippina would show, along with their edges. If we set the depth to 2, it would be all nodes regardless of node label or type that are connected to Agrippina, plus nodes with to those 1st-depth nodes, along with all of the edges for both the 1st layer of direct-connect nodes and the 2nd layer of nodes connected to Agrippina only indirectly. A broader node-type example: If we’ve selected node-type “person” with a depth of 1, any node connected to a node with the type “person” would show, along with its edges. There’s a good walkthrough of ego networks at https://blog.ouseful.info/2010/05/10/getting-started-with-gephi-network-visualisation-app-–-my-facebook-network-part-iii-ego-filters-and-simple-network-stats/ <https://blog.ouseful.info/2010/05/10/getting-started-with-gephi-network-visualisation-app-%E2%80%93-my-facebook-network-part-iii-ego-filters-and-simple-network-stats/> and Kumu has a “focus” button that lets you see 1-level depth context for a node you’ve chosen. That said, context display may be beyond what’s possible for this round of filtering. If it’s doable, that’d be great, but the primary reason for calling it out here is to make sure that it’s on your radar as you think about what this round of node/edge-type or text-search filtering will need to support in the next round. —k

…

On Aug 11, 2020, at 12:06 PM, benloh ***@***.***> wrote: @jdanish <https://github.com/jdanish> @kalanicraig <https://github.com/kalanicraig> So...filtering is anything but simple. While a basic data filtering algorithm is fairly easy, multi-field filtering and a UI that is dynamically generated from the template is fairly complex. Two questions about the filtering design: Given how much room the filter takes up, I figured it was easier putting it in the info panel/tab bar along with everything else. That way users won't have to manage an extra window. Is that OK? Here's a rough first pass. It can be tightened up. <https://user-images.githubusercontent.com/1403057/89919726-80cf0780-dbb0-11ea-9237-b23771316768.png> When you set a filter are you saying "Search for and show me nodes that match xyz"? or are you saying "Search for and hide nodes that match xyz"? I was finding myself somewhat confused by it as I was playing with it. At first I expected it to behave like the "Search" function. Searching for a label that "contains" C means show me all the labels that have C. But I imagine whether it's show or hide depends on the nature of the filtering you're wanting to do. Do you have any examples of the types of filtering you want to do / have done on existing graphs? Should it be show or hide? Or does that actually need to be a separate command option for the whole filter? e.g. at the top we might have a drop down that says "Show all nodes that match" vs "Hide all nodes that match"? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#92 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACKL4NBR4YPRLLNYD6EL42LSAFT7ZANCNFSM4N6SDGGQ>.

[jdanish]

Agreed. One of the tensions we are facing as a design team are that the person who doesn’t know much about networks (me) is thinking of this in terms of general data / database functions that I think will benefit students. The person who does understand networks (Kalani) is thinking about it in terms of network vis ideas / concepts. We ultimately want to support both in interesting ways that support learners and that go beyond what others have done. For the fall Net.Create implementation that Kalani is doing on the original grant, she might use the context, but is also fine without filtering. For me testing future ideas I think the filtering will help with middle school kids (possible pilot) and / or my grad class (also possible study to show future applications). In the future, you can likely divide it up into the grants we propose with…. 1) Follow up via NSF for more undergrad classes highlighting the context feature 2) NEH-style grants for practitioners definitely highlighting common network approaches (context) (though Joshua thinks if we do it right hey’ll dig the filtering) 3) Follow up to the Spencer / other NSF for middle school might use both and likely will explore something else that is more wizard like (maybe). But I think whatever we can do now will help us think about all of those future options nicely.

[jdanish]

Two other quick thoughts:

1. Another use we imagined for filtering based on dates is similar to a cheap version of a future timeline view. One day we hope to see how the network changes over time, but for now if we can filter to show all of the nodes with labels of a certain date or later, then we can see that slice in time (and the edges that connect them, of course).

2. Related to the show question - I think for now just "show" is fine. Long-term, it'd be nice to choose between the show that implies remove the rest and "highlight" which basically fades the rest so that the specific network elements pop up. There is an example of that here, if you click the choices on the top: https://kumu.io/tnc/tnc-global-situation-analysis-v2#tnc-gsa-map.

I'm assuming this is all "for later" but wanted to archive it and share in case it impacts your thinking.

[jdanish]

Highlighting demonstrated in the above link:

[benloh]

Thanks for the details. That's really helpful. Let me try to tease this apart.

1. Tab Panel Approach

We'll add a line showing the current filter. Probably at the top of the graph or bottom of the tab panel.

2. Ego Networks

We do currently have a poor man's version of this: If you hover over a node, you'll see the connected nodes at a depth of 1.

I can see how it'd be cool to be able to set different depths. I'm guessing this is something we'd want to more carefully design with a future grant. e.g. how do you set the depth, does it show on select or hover or both,

That said, it is fundamentally a different animal under the hood than the other filters.

3. Date Filtering

Yeah, I was thinking date filtering would be cool. We'd just have to figure out a good UI for it. The actual filtering is relatively straightforward, but making it easy to flexibly enter a valid date is harder. Hopefully there's a library we can use.

4. Show vs Highlight

This is probably relatively easy to implement. The way we're hiding filtered elements is setting their opacity. I can add this if we have budget. But if we need to reserve some for MEME, this is probably out.

Which actually brings up the next point...

5. Hiding vs Removing

I didn't want to overwhelm the previous conversation, but my next big design question would have been around hiding vs removing the filtered elements. I was figuring that hiding them would work better because the whole network would not completely reconfigure itself when you turned on a filter. Instead, you'd see the nodes and edges fade away. The alternative, removing, would cause the whole network to jump and reassemble itself, and you'd completely lose the sense of where you are in space and what you're looking at. I assume that's the right approach. I'll try to have something you can play with soon to verify.

[kalanicraig]

Given all of these things, I think hiding is the right answer for now and for the foreseeable future. Network scientists can import into Gephi to reconfigure the subnetwork with new stats to match only the things that show. Our novice and intermediate users, on the other hand, will be using this to find the nodes they’re interested in in situ and then once the whole network is showing again, use that to keep their eye on those nodes and edges now. —k

…

On Aug 12, 2020, at 12:21 PM, benloh ***@***.***> wrote: Thanks for the details. That's really helpful. Let me try to tease this apart. Tab Panel Approach We'll add a line showing the current filter. Probably at the top of the graph or bottom of the tab panel. Ego Networks We do currently have a poor man's version of this: If you hover over a node, you'll see the connected nodes at a depth of 1. I can see how it'd be cool to be able to set different depths. I'm guessing this is something we'd want to more carefully design with a future grant. e.g. how do you set the depth, does it show on select or hover or both, That said, it is fundamentally a different animal under the hood than the other filters. Date Filtering Yeah, I was thinking date filtering would be cool. We'd just have to figure out a good UI for it. The actual filtering is relatively straightforward, but making it easy to flexibly enter a valid date is harder. Hopefully there's a library we can use. Show vs Highlight This is probably relatively easy to implement. The way we're hiding filtered elements is setting their opacity. I can add this if we have budget. But if we need to reserve some for MEME, this is probably out. Which actually brings up the next point... Hiding vs Removing I didn't want to overwhelm the previous conversation, but my next big design question would have been around hiding vs removing the filtered elements. I was figuring that hiding them would work better because the whole network would not completely reconfigure itself when you turned on a filter. Instead, you'd see the nodes and edges fade away. The alternative, removing, would cause the whole network to jump and reassemble itself, and you'd completely lose the sense of where you are in space and what you're looking at. I assume that's the right approach. I'll try to have something you can play with soon to verify. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#92 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACKL4NBC3PSYP4NNFAOV7PTSAK6RHANCNFSM4N6SDGGQ>.

[jdanish]

Just wanted to add:

1. Don't worry about reserving $ for MEME - we'll handle that separately.
2. I agree to do hiding (instead of removing) if we have to choose, but also see the value of having that be a flag if possible
3. For the dates, a library and validation long-term would be great, but for now if we can convince them to use a decent format (and then clean it up as needed), and then we just do normal less / more won't it be handled naturally? We'd have to do cleanup for the inevitable kid who puts a date in the wrong, but otherwise, I think that'd work. So, for example, if the format is YYYY/MM/DD then 2019/01/01 is less than 2020/01/01 and 2020/01/01 is less than 2020/01/02 using basic string comparison, no? Naturally, kids not putting leading zeros causes problem that we'll want to solve later with validation and a library etc., but short term we can edit.

Incidentally, the timeline / dates is where I imagine actually removing nodes that are not yet present is really powerful because you'd see the node sizes change over time as they shift who they are connected to.

[benloh]

Hmm...I hadn't thought of the fact that node size changes. That has big implications for how we're handling this filtering because then we're talking about modifying the underlying data model and keeping track of a local "play/display-only" dataset vs the shared common dataset. That's a pretty big change in how we handle data. And very much a future feature.

Regarding date, I think treating the date as a string would be too brittle. And it requires that you have good data in the first place. Are we actually using a date field at the moment?

[jdanish]

Gotcha. Then hiding for now and making sure we get that next grant! :)

…

---- (from my iPhone) Joshua Danish http://www.joshuadanish.com

On Aug 12, 2020, at 2:57 PM, benloh ***@***.***> wrote:  Hmm...I hadn't thought of the fact that node size changes. That has big implications for how we're handling this filtering because then we're talking about modifying the underlying data model and keeping track of a local "play/display-only" dataset vs the shared common dataset. That's a pretty big change in how we handle data. And very much a future feature. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[benloh]

Implemented with #113.

[benloh]

Date filtering logged in #122.