ipywidgets cannot render when connected to a kernel of another notebook

[nulinspiratie]

I am having issues displaying ipywidgets when two notebooks share the same kernel.
The original notebook (which instantiated the kernel) is still able to display widgets.
However, if I connect another notebook to the same kernel, the second notebook is unable to display any widgets. Instead, an error message is displayed (see below for details).
Oddly enough, the original notebook can still display widgets without any issues.

The issue is somewhat related to #1218 and #1881 although the issues are fairly old, and it sounds as though it should be possible to use ipywidgets when a kernel is shared by multiple frontends.

Is this a known issue, and is there any solution to this?

Steps to reproduce:

1. Create notebook1.ipynb and initialize kernel.
2. Create notebook2.ipynb and initialize kernel.
3. Change kernel of notebook2.ipynb to that of notebook1.ipynb
4. Run the following code on notebook2.ipynb:

from ipywidgets.widgets import Button
Button()

Expected result:

An ipywidget button appears

Actual results

Output shows Error displaying widget: model not found along with the following error message in the console:

System specs:

OS Windows 10
python 3.7
ipython 7.9.0
ipywidgets 7.5.1
jupyter 1.0.0
jupyter-client 5.3.4
jupyter-console 6.0.0
jupyter-core 4.6.1
jupyterlab 1.2.3
jupyterlab-server 1.0.6
NodeJS 12.13.0

$ jupyter labextension list
JupyterLab v1.2.3
Known labextensions:
app dir: c:\programdata\anaconda3\share\jupyter\lab
@jupyter-widgets/jupyterlab-manager v1.1.0 enabled ok
@jupyter-widgets/jupyterlab-sidecar v0.4.0 enabled ok
bqplot v0.5.1 enabled ok
jupyter-matplotlib v0.4.2 enabled ok

[nulinspiratie]

Is there anyone who knows if ipywidgets are or are not meant to work with a shared kernel?

If they are, any idea if the issue outlined above might have a simple fix, or that it hits at a fundamental ipywidgets limitation? If someone can point me in the right direction it would help me to look further into it

[jasongrout]

Unfortunately, the underlying comm message spec has some flaws when you have multiple clients talking to the same kernel, so no, widgets can't be reliably used on multiple simultaneous notebooks. I would love to see these flaws fixed, but it is going to take a change to the Jupyter protocol. See jupyter/jupyter_client#263 for more details.

[jasongrout]

One shortcut that we have built in to JLab is that if you have a single notebook and make two views of that same notebook, or "Create new output view" of some of the cells, it will use the same connection and same underlying widget models, and widgets will work in both tabs.

[nulinspiratie]

@jasongrout I have looked a bit more into this issue, and from what I can tell the underlying issue is that each notebook has its own WidgetManager. However, when a new widget is created (regardless of the notebook), it is always registered with the WidgetManager whose notebook started the shared kernel.

As a test, I manually updated WidgetManager._models of the second notebook to equal that of the first notebook (using Chrome DevTools). After doing so, the second notebook had no problems whatsoever displaying the widget. I'm therefore thinking, is there a way for a WidgetManager to check if there are other WidgetManagers that share the same kernel, and if so, broadcast the creation of a new widget? I think this would solve the problem.

I can currently only think of hacky solutions, would there be a straightforward proper fix?

[jasongrout]

is there a way for a WidgetManager to check if there are other WidgetManagers that share the same kernel, and if so, broadcast the creation of a new widget? I think this would solve the problem.

Or perhaps use a single widget manager per kernel, rather than a single widget manager per kernel connection (i.e., each document here has a separate kernel connection, but to the same kernel). I think that could maybe work, but we'd have to check ramifications since the widget manager also deals with the notebook, for example state is restored from the notebook document too, and having two different notebook documents share the same kernel manager would complicate that.

[jasongrout]

Setting as future as an idea to explore when someone has time. If you want to explore these ideas in a PR, we can help you get started.

[nvaytet]

Hi, is there any news on this? Thanks.

[maartenbreddels]

Yes, version 8.0 by default will echo messages, which will support multiple clients (added in #3195 )

However, I am not 100% sure it addresses the original issue of this issue. Can you try?