MRG: Prototype of notebook viz (ipyvtk) #8503

GuillaumeFavelier · 2020-11-10T16:05:36Z

This PR updates the current notebook 3d backend to use ipyvtk. It allows native support for widgets, 3d render view and intuitive camera interactions. It could also support google colab and jupyter lab? More test is needed.

It's still a work in progress:

code

import os
import mne
from mne.datasets import sample
mne.viz.set_3d_backend('notebook')
data_path = sample.data_path()
sample_dir = os.path.join(data_path, 'MEG', 'sample')
subjects_dir = os.path.join(data_path, 'subjects')
fname_stc = os.path.join(sample_dir, 'sample_audvis-meg')
stc = mne.read_source_estimate(fname_stc, subject='sample')
initial_time = 0.13
brain = stc.plot(subjects_dir=subjects_dir, initial_time=initial_time,
                 clim=dict(kind='value', pos_lims=[3, 6, 9]),
                 time_viewer=True,
                 show_traces=True,
                 hemi='split',
                 size=600,
                 background="white",
                 )

hoechenberger · 2020-11-11T09:44:22Z

😍😍😍

GuillaumeFavelier · 2020-12-16T09:54:09Z

Update after 4d9b63a:

I think we could use ipywidgets to reproduce the tool bar buttons.

I started by working directly on Brain but next I'll see how the other 3d plots behave.

GuillaumeFavelier · 2020-12-16T13:08:11Z

Update after dcaff44:

larsoner · 2020-12-16T13:27:33Z

Lots of - lines is nice, and it seems to work well! I assume this is server-side rendering piping PNGs to the client?

Anything else left to do here?

@christianhacker is this more like what you want to use?

larsoner · 2020-12-16T13:28:43Z

Anything else left to do here?

I guess one thing from seeing what you've shown so far would be making sure that show_traces=True adds a matplotlib plot below. Would this require a lot of hacking of the backend to get right? Maybe this is a PR we can merge early in the 0.23 cycle so people can play with it.

GuillaumeFavelier · 2020-12-16T13:57:58Z

I assume this is server-side rendering piping PNGs to the client?

After digging into https://github.com/Kitware/ipyvtk-simple/blob/master/ipyvtk_simple/viewer.py#L163-L171 I can confirm this indeed. JPEG though :)

Anything else left to do here?

shortcuts
toolbar
matplotlib canvas

Would this require a lot of hacking of the backend to get right?

I don't think so...

GuillaumeFavelier · 2020-12-16T14:51:58Z

Update after ec0d265:

larsoner · 2020-12-16T14:54:50Z

mne/viz/_brain/mplcanvas.py

-        self.axes = self.fig.add_subplot(111)
+        self.notebook = notebook
+        if self.notebook:
+            self.fig, self.axes = plt.subplots()


if you set the figsize and dpi and such it will even be the right size! Not 100% sure how you get it to show up below the brain plot, probably has to do with when each is instantiated which seems like it could be a pain to manage

I don't know where to get correct values for dpi. We were using Qt for that before

I would try just setting it to 100. I don't know if it will matter. If it does we can figure out the right thing to do. One reason you need it is because there figsize is in inches, so you need to know the DPI to get from the pixel width to the figsize.

christianhacker · 2020-12-16T14:55:22Z

It's beautiful...

I'll test it out when I have more time, but this looks very good. Thank you for your hard work.

larsoner · 2020-12-16T14:55:41Z

For shortcuts I'm not sure where to look. I'd see how matplotlib handles it.

For toolbar I'd also look at how matplotlib makes theirs and use the same code

GuillaumeFavelier · 2020-12-17T10:04:01Z

Update after c254d79:

The matplotlib figure events are now connected correctly.

Supported shortcuts:

i toggle slider visibility
s apply auto scaling
r restore scaling
c clear glyphs

Not supported (yet):

play (this one was based on QThread, I need a workaround)
? help (just does not work for me)
ctrl+shift+s save movie (based on modifiers, I need to find a solution too)

agramfort · 2021-01-04T12:26:20Z

please update what's new page and then +1 for MRG when CIs are green

agramfort

+1 to merge when green !

agramfort · 2021-01-04T16:32:17Z

ok to merge for you @larsoner ?

mne/viz/_brain/_brain.py

mne/viz/_brain/mplcanvas.py

mne/viz/_brain/_brain.py

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

larsoner

LGTM +1 for merge once CIs are happy

agramfort · 2021-01-06T16:33:11Z

🍻 🍾 🎉 !!!

@GuillaumeFavelier can you tweet a screenshot movie of this?

GuillaumeFavelier · 2021-01-06T17:01:01Z

Unfortunately, I'm not on Twitter 😅

I can still prepare a video though 👍

GuillaumeFavelier · 2021-01-06T17:13:14Z

Here is a simple MP4 demo :

output.mp4

drammock · 2021-01-06T17:30:30Z

Unfortunately, I'm not on Twitter sweat_smile

I think you mean fortunately I'm not on twitter :)

agramfort · 2021-01-06T17:38:38Z

that will do it thanks !

…

* upstream/master: MAINT: Actually use all caches (mne-tools#8702) MRG: Prototype of notebook viz (ipyvtk) (mne-tools#8503) MRG: Use caching in Github Actions and Azure Pipelines (mne-tools#8695)

* Deploy basic version * Update prototype [skip ci] * Update _Renderer [skip ci] * Update picking [skip ci] * Fix disp * Add shortcuts [skip ci] * Update tests * Add ipyvtk_simple * Fix style * Update tests * Refactor shortcuts * Improve tests * Remove cruft * Remove cruft * Reorder figures * Add tool bar * Reorder figures * Fix visibility * Improve coverage * Fix ratio and layout * Update changes * Do not import pyplot * Update mne/viz/_brain/_brain.py Co-authored-by: Eric Larson <larson.eric.d@gmail.com> * Use _add_action * Use icon + tooltip * Fix qt func * Reorder actions * Switch to icon + description Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

Deploy basic version

50789de

GuillaumeFavelier self-assigned this Nov 10, 2020

GuillaumeFavelier added the VIZ label Nov 12, 2020

Merge branch 'master' into proto/ipyvtk_backend

cfd0678

GuillaumeFavelier mentioned this pull request Dec 15, 2020

Headless container with mne pyvista backend #8659

Closed

GuillaumeFavelier added 2 commits December 16, 2020 10:24

Merge branch 'master' into proto/ipyvtk_backend

d7d2c5b

Update prototype [skip ci]

4d9b63a

Update _Renderer [skip ci]

dcaff44

Update picking [skip ci]

ec0d265

larsoner reviewed Dec 16, 2020

View reviewed changes

GuillaumeFavelier added 2 commits December 16, 2020 16:11

Fix disp

9a22fb5

Add shortcuts [skip ci]

c254d79

GuillaumeFavelier added 8 commits December 17, 2020 11:23

Update tests

a3fb818

Add ipyvtk_simple

909909c

Fix style

f317572

Update tests

08fdd85

Refactor shortcuts

040bc3d

Improve tests

4cac2a0

Remove cruft

a789b46

Remove cruft

fd3183b

Merge branch 'master' into proto/ipyvtk_backend

1cc4757

Update changes

b16738c

agramfort approved these changes Jan 4, 2021

View reviewed changes

larsoner reviewed Jan 4, 2021

View reviewed changes

mne/viz/_brain/_brain.py Outdated Show resolved Hide resolved

larsoner reviewed Jan 4, 2021

View reviewed changes

mne/viz/_brain/mplcanvas.py Outdated Show resolved Hide resolved

Do not import pyplot

195c91b

larsoner reviewed Jan 5, 2021

View reviewed changes

mne/viz/_brain/_brain.py Outdated Show resolved Hide resolved

GuillaumeFavelier and others added 7 commits January 5, 2021 14:55

Update mne/viz/_brain/_brain.py

ed680d3

Co-authored-by: Eric Larson <larson.eric.d@gmail.com>

Use _add_action

57b505f

Use icon + tooltip

4059082

Fix qt func

af7f988

Merge branch 'master' into proto/ipyvtk_backend

b4b49f0

Reorder actions

a5e44e7

Switch to icon + description

0dcc7f6

larsoner approved these changes Jan 6, 2021

View reviewed changes

GuillaumeFavelier merged commit cab1425 into mne-tools:master Jan 6, 2021

GuillaumeFavelier deleted the proto/ipyvtk_backend branch January 6, 2021 16:24

GuillaumeFavelier mentioned this pull request Jan 7, 2021

ENH: Notebook 3d backend improvements (ipyvtklink) #8704

Closed

11 tasks

larsoner mentioned this pull request Feb 7, 2021

Support for use of alternate atlases (e.g. Human Connectome Project atlas) nipy/PySurfer#134

Closed

GuillaumeFavelier mentioned this pull request Sep 3, 2021

Displaying ipycanvas in google colab? jupyter-widgets-contrib/ipycanvas#170

Closed

larsoner mentioned this pull request Jun 17, 2022

[MAINT] Remove redundancy in Abtract Qt Classes #10779

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MRG: Prototype of notebook viz (ipyvtk) #8503

MRG: Prototype of notebook viz (ipyvtk) #8503

GuillaumeFavelier commented Nov 10, 2020 •

edited

Loading

hoechenberger commented Nov 11, 2020

GuillaumeFavelier commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

larsoner commented Dec 16, 2020

larsoner commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

larsoner Dec 16, 2020

GuillaumeFavelier Dec 16, 2020

larsoner Dec 16, 2020

christianhacker commented Dec 16, 2020

larsoner commented Dec 16, 2020

GuillaumeFavelier commented Dec 17, 2020

agramfort commented Jan 4, 2021

agramfort left a comment

agramfort commented Jan 4, 2021

larsoner left a comment

agramfort commented Jan 6, 2021

GuillaumeFavelier commented Jan 6, 2021

GuillaumeFavelier commented Jan 6, 2021

drammock commented Jan 6, 2021

agramfort commented Jan 6, 2021 via email

MRG: Prototype of notebook viz (ipyvtk) #8503

MRG: Prototype of notebook viz (ipyvtk) #8503

Conversation

GuillaumeFavelier commented Nov 10, 2020 • edited Loading

hoechenberger commented Nov 11, 2020

GuillaumeFavelier commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

larsoner commented Dec 16, 2020

larsoner commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

GuillaumeFavelier commented Dec 16, 2020

larsoner Dec 16, 2020

Choose a reason for hiding this comment

GuillaumeFavelier Dec 16, 2020

Choose a reason for hiding this comment

larsoner Dec 16, 2020

Choose a reason for hiding this comment

christianhacker commented Dec 16, 2020

larsoner commented Dec 16, 2020

GuillaumeFavelier commented Dec 17, 2020

agramfort commented Jan 4, 2021

agramfort left a comment

Choose a reason for hiding this comment

agramfort commented Jan 4, 2021

larsoner left a comment

Choose a reason for hiding this comment

agramfort commented Jan 6, 2021

GuillaumeFavelier commented Jan 6, 2021

GuillaumeFavelier commented Jan 6, 2021

drammock commented Jan 6, 2021

agramfort commented Jan 6, 2021 via email

GuillaumeFavelier commented Nov 10, 2020 •

edited

Loading