[MRG] NetworkPlot class to enable interactive visualizations and animations #649

ntolley · 2023-05-22T21:53:59Z

Here's my first pass at creating a viz class that can be used to quickly render the full 3D network, as well as membrane potentials of individual sections. This is meant to be largely generic and flexible so that it can be co-opted for other purposes. I think the most interesting applications at the moment would be:

Quickly generating animations of HNN simulations
Embedding the 3D plot into the current ipywidgets GUI

The current strategy for the class is to do all of the heavy lifting at the initialization (namely creating the 3D object, and mapping the voltage recordings to the appropriate colors). For a quick demo, the code below can be run on my laptop on a single core in approximately 20 sec:

from hnn_core import jones_2009_model, simulate_dipole
from hnn_core.network_models import add_erp_drives_to_jones_model
from hnn_core.viz import NetworkPlot

net = jones_2009_model()
net.set_cell_positions(inplane_distance=300)
add_erp_drives_to_jones_model(net)
dpl = simulate_dipole(net, dt=0.5, tstop=170, record_vsec='all')

net_plot = NetworkPlot(net)

If you want to play around with visualizing the voltages, you can try the following code block:

from ipywidgets import interact, IntSlider

def update_plot(t_idx, elev, azim):
    net_plot.update_section_voltages(t_idx)
    net_plot.elev = elev
    net_plot.azim = azim
    return net_plot.fig

time_slider = IntSlider(min=0, max=len(net_plot.times), value=1, continuous_update=False)
elev_slider = IntSlider(min=-100,max=100, value=10, continuous_update=False)
azim_slider = IntSlider(min=-100,max=100, value=-100, continuous_update=False)

interact(update_plot, t_idx=time_slider, elev=elev_slider, azim=azim_slider)

ntolley · 2023-05-22T22:24:48Z

@jasmainak @rythorpe perhaps it'd be a good idea to make this a separate file? If we can find a solution to updating the section colors very rapidly, we can even consider adding a "play" button inside the GUI (or on the HNN website!) like the example here: https://matplotlib.org/stable/gallery/animation/unchained.html

@chenghuzi you might have some good insight on how to make the current code more "reactive"

jasmainak · 2023-05-22T22:27:21Z

examples/howto/plot_hnn_animation.py

@@ -0,0 +1,37 @@
+"""
+================================
+XX. Modifying local connectivity


to be fixed :)

jasmainak · 2023-05-22T22:28:05Z

examples/howto/plot_hnn_animation.py

+elev_slider = IntSlider(min=-100, max=100, value=10, continuous_update=False)
+azim_slider = IntSlider(min=-100, max=100, value=-100, continuous_update=False)


isn't ipympl a dependency of hnn-gui ? Do you still need this?

codecov-commenter · 2023-05-22T22:28:13Z

Codecov Report

Attention: Patch coverage is 95.17544% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 91.51%. Comparing base (819f4f4) to head (db60ec2).
Report is 1 commits behind head on master.

❗ Current head db60ec2 differs from pull request most recent head e9a94ef

Please upload reports for the commit e9a94ef to get more accurate results.

Files	Patch %	Lines
hnn_core/viz.py	95.13%	11 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #649      +/-   ##
==========================================
- Coverage   92.33%   91.51%   -0.82%     
==========================================
  Files          27       25       -2     
  Lines        5059     4819     -240     
==========================================
- Hits         4671     4410     -261     
- Misses        388      409      +21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

hnn_core/viz.py

jasmainak · 2023-05-22T22:33:39Z

Agnostic about saving to a different file ... see my comment about the azimuth/elevation control. Maybe it makes the class thinner?

jasmainak · 2023-05-22T22:34:04Z

hnn_core/viz.py

@@ -1185,3 +1198,234 @@ def plot_laminar_csd(times, data, contact_labels, ax=None, colorbar=True,
    plt_show(show)

    return ax.get_figure()
+
+
+class NetworkPlot:


Suggested change

class NetworkPlot:

class NetworkDataPlotter:

?

NetworkPlotter? It might useful too for just visualizing the 3D morphology without any data too

Yeah if data is None

jasmainak · 2023-05-22T22:36:40Z

hnn_core/viz.py

+        self.init_network_plot()
+        self._update_axes()
+
+    def get_voltages(self):


wondering if this should be in Network class ... then you can pass data to the NetworkDataPlotter class. You can plot not just voltages but other quantities

ntolley · 2023-05-23T22:56:52Z

Dropping this here for easy testing once we get closer to merging. I think rendering movies is going to be a longish wait for larger networks, but not absurdly so (maybe like 5-10 minutes? if you downsample?)

import hnn_core
import os.path as op
from hnn_core import jones_2009_model, simulate_dipole, read_params
from hnn_core.network_models import add_erp_drives_to_jones_model
from hnn_core.viz import NetworkPlot

hnn_core_root = op.dirname(hnn_core.__file__)
params_fname = op.join(hnn_core_root, 'param', 'default.json')
params = read_params(params_fname)
params.update({'N_pyr_x': 3, 'N_pyr_y': 3})
net = jones_2009_model(params)
net.set_cell_positions(inplane_distance=300)
add_erp_drives_to_jones_model(net)
dpl = simulate_dipole(net, dt=0.5, tstop=170, record_vsec='all')

net_plot = NetworkPlot(net)
net_plot.export_movie('demo.gif', dpi=200)

ntolley · 2023-05-23T23:01:49Z

hnn_core/viz.py

+
+        self.ax.view_init(self._elev, self._azim)
+
+    def export_movie(self, fname, dpi=300):


@jasmainak here's the export function!

There are some solutions here that might speed up the exporting by circumventing the matplotlib animation module:
https://stackoverflow.com/questions/4092927/generating-movie-from-python-without-saving-individual-frames-to-files

Is it possible to add a progressbar using a callback? I don't think the speed is a showstopper here. Maybe a param called "decim" could be added to take every nth frame. Make it work, make it nice, make it fast ...

Definitely would be a nice addition but I think I'd need to be much more matplotlib savvy to know how to implement that 😄

chenghuzi · 2023-05-28T20:20:32Z

@jasmainak @rythorpe perhaps it'd be a good idea to make this a separate file? If we can find a solution to updating the section colors very rapidly, we can even consider adding a "play" button inside the GUI (or on the HNN website!) like the example here: https://matplotlib.org/stable/gallery/animation/unchained.html

@chenghuzi you might have some good insight on how to make the current code more "reactive"

Great work! Although, I'm concerned about the reactivity of our GUI. We are taking snapshots from the buffer and using imshow to render network images for now. So, it might be difficult to have real-time interaction in the new GUI with these new settings. But this will not affect the current function so there is nothing else to worry about.

hnn_core/viz.py

rythorpe · 2023-09-22T21:03:10Z

examples/howto/plot_hnn_animation.py

+###############################################################################
+from hnn_core import jones_2009_model, simulate_dipole
+from hnn_core.network_models import add_erp_drives_to_jones_model
+from hnn_core.viz import NetworkPlot


Suggested change

from hnn_core.viz import NetworkPlot

from hnn_core.viz import NetworkPlotter

I'm guessing the name of this class got changed somewhere along the way and you forgot to update it here?

rythorpe · 2023-09-22T21:06:00Z

examples/howto/plot_hnn_animation.py

+    net_plot.azim = azim
+    return net_plot.fig
+
+time_slider = IntSlider(min=0, max=len(net_plot.times), value=1, continuous_update=False)


For some reason the interactive slider didn't work for me. It might be a version issue and totally unrelated to your code here. Have you encountered a similar issue at any point @ntolley?

ntolley · 2023-09-25T17:59:27Z

@rythorpe @jasmainak this PR is ready for review! Two weird errors have come up however. First it seems like sphinx-gallery doesn't support python versions older than 3.8

Does this mean our minimum python version needs to be updated?

Also the windows unit tests are failing with an error in the GUI code...

This is the offending line in case you guys have any ideas on the possible cause:
https://github.com/ntolley/hnn-core/blob/e3aa27c91ba4ed7478d2214e53fea618b793f45b/hnn_core/tests/test_viz.py#L23C5-L23C5

ntolley · 2023-09-25T18:03:08Z

Yup just confirmed, sphinx now requires python>=3.8

https://github.com/sphinx-gallery/sphinx-gallery/blob/e2903c9e43572a0a29850f7ab153ff69098f5c81/setup.py#L58C5-L58C5

rythorpe · 2023-09-25T18:15:19Z

.circleci/config.yml

@@ -27,7 +27,7 @@ jobs:
              command: |
                  export PATH=~/miniconda/bin:$PATH
                  conda update --yes --quiet conda
-                  conda create -n testenv --yes pip python=3.7
+                  conda create -n testenv --yes pip python=3.8


Do you think we need to update setup.py and our unit tests with python>=3.8 as well to ensure all builds are getting tested properly?

Honestly if this only impacts the documentation building I don't think it's necessary

that's true but you won't realize if other dependencies start requiring 3.8 now that you have updated it here. I don't think we have a CI that tests for python < 3.8? Better to be consistent and update readme/setup etc.

I went ahead and updated this in #677. We'll see if it fixes the Windows test_viz.py error....

excellent!!

rythorpe · 2023-09-25T18:21:59Z

@rythorpe @jasmainak this PR is ready for review! Two weird errors have come up however. First it seems like sphinx-gallery doesn't support python versions older than 3.8

Does this mean our minimum python version needs to be updated?

Also the windows unit tests are failing with an error in the GUI code...

This is the offending line in case you guys have any ideas on the possible cause: https://github.com/ntolley/hnn-core/blob/e3aa27c91ba4ed7478d2214e53fea618b793f45b/hnn_core/tests/test_viz.py#L23C5-L23C5

Ugh, I'm guessing the most recent version of matplotlib has diverged for windows vs. linux. Something to do with the agg backend....

ntolley · 2023-09-25T19:08:02Z

Ok updating the python version used for CircleCI worked, however it seems something in the MNE code is breaking the somato example now:

examples/howto/plot_hnn_animation.py

hnn_core/viz.py

examples/howto/plot_hnn_animation.py

hnn_core/tests/test_viz.py

Co-authored-by: Mainak Jas <jasmainak@users.noreply.github.com>

doc/whats_new.rst

rythorpe · 2024-05-23T00:20:49Z

Aside from @jasmainak's spelling error and my incredibly nitpicky punctuation suggestion above, this looks great @ntolley !

hnn_core/viz.py

Co-authored-by: Ryan Thorpe <ryvthorpe@gmail.com>

rythorpe · 2024-05-23T18:12:52Z

I've enabled auto-merge! Thanks @ntolley for this incredible feature - I'm very excited to put it to good use 😄

jasmainak reviewed May 22, 2023

View reviewed changes

hnn_core/viz.py Show resolved Hide resolved

jasmainak reviewed May 22, 2023

View reviewed changes

ntolley commented May 23, 2023

View reviewed changes

ntolley force-pushed the hnn_animation branch from 6a8e90f to 0c1b6fe Compare June 25, 2023 00:53

ntolley force-pushed the hnn_animation branch from 525c749 to 3c0bccc Compare September 21, 2023 13:44

rythorpe reviewed Sep 22, 2023

View reviewed changes

hnn_core/viz.py Outdated Show resolved Hide resolved

rythorpe reviewed Sep 22, 2023

View reviewed changes

ntolley changed the title ~~WIP: NetworkPlot class to enable interactive visualizations and animations~~ [MRG] NetworkPlot class to enable interactive visualizations and animations Sep 25, 2023

rythorpe reviewed Sep 25, 2023

View reviewed changes