Differing behaviors between classic Jupyter Notebook and JupyterLab & VS Code for ipywidgets

[mwh3780]

Environment data

• VS Code version: 1.58.2 (Universal)
• Jupyter Extension version (available under the Extensions sidebar): v2021.8.1046824664
• Python Extension version (available under the Extensions sidebar): v2021.6.944021595
• OS (Mac) and version: macOS Big Sur (11.4)
• Python and/or Anaconda version: Python 3.6.12 |Anaconda, Inc.| (default, Sep 8 2020, 17:50:39) [GCC Clang 10.0.0 ]
• Type of virtual environment used (conda): See above
• Jupyter server running: Local

Expected behaviour

Updating or interacting with widgets triggers handlers which sends output to cell

Actual behaviour

Included in summary

Steps to reproduce:

import ipywidgets
import IPython

import numpy as np
import pandas as pd

import seaborn as sns

np.random.seed(1)
df_1 = pd.DataFrame(np.random.randint(1, 5, (10, 5))).add_prefix("X")

def getHist(df, var_name):
    import matplotlib.pyplot as plt
    table = df
    fig, ax = plt.subplots()
    try:
        sns.histplot(table[var_name])
    except:
        sns.distplot(table[var_name])
    IPython.display.display(fig)
    plt.close()

class UI():
    def __init__(self):
        self._select_widget = ipywidgets.Dropdown(options = df_1.columns.tolist())
        self._select_widget.observe(self._on_value_change, names = "value")
        
        self._print_Data_syntax_button = ipywidgets.Button(description = "Data Syntax")
        self._print_Data_syntax_button.on_click(self._on_button_clicked_print_Data_syntax)    
        
        self._vbox = ipywidgets.VBox([self._print_Data_syntax_button,
                                      self._select_widget,
                                     ])
        
        self._display()
        
    def _on_value_change(self, change):
        self._display()
        
    def _displayUI(self):
        IPython.display.clear_output()
        IPython.display.display(self._vbox)
        
    def _display(self):
        self._displayUI()
        getHist(df_1, self._select_widget.value)
        
    def _on_button_clicked_print_Data_syntax(self, change):
        print(str(self._select_widget.value))

ui1 = UI()
# Pushing button or changing dropdown triggers behavior noted in Summary

Logs

Included in summary

Summary

I recently inherited some code from someone and in the process of trying to run it in a classic Jupyter Notebook and in JupyterLab & VS Code, I noticed it behaves slightly different in each. Below is stripped down minimal code to reproduce the behavior I'm seeing, and I've preserved the structure of the code (my disclaimer that it’s a bit odd and not optimal) in case that comes into play.
In a classic Jupyter Notebook, when either the button is clicked or the dropdown changes new output is produced and sent to the output of the cell. But in JupyterLab the output is sent to the Log and the output in the cell stays the same. This also happens when I use the same notbook server for the classic notebook that I'm using for JupyterLab. Interestingly VS Code has similar behavior as JupyterLab in that the output in the cell stays the same and warnings/info messages are thrown. Also if it helps, in VS Code, in the Output - Jupyter section it gives the following warnings/info messages when the dropdown widget is changed:

Info 2021-07-19 10:36:07: Unhandled widget kernel message: display_data [object Object]
Unhandled kernel message from a widget: display_data : {"data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUg [truncated]

and the following when the button is clicked:

Info 2021-07-19 10:38:18: Unhandled widget kernel message: stream [object Object]
Unhandled kernel message from a widget: stream : {"name":"stdout","text":"X4\n"}

My hunch is that this isn't a bug per-se but that, but rather that the code improperly relies on behavior that differs between a classic Jupyter Notebook and JupyterLab & VS Code. Any ideas as to what the issue may be or what the differing behaviors are and why it works in a classic Jupyter Notebook but not JupyterLab / VS Code?

Thanks

Setup versioning:
Python 3.6.12 |Anaconda, Inc.| (default, Sep 8 2020, 17:50:39)
notebook 6.4.0
jupyterlab 3.0.16
ipywidgets 7.6.3
IPython 7.16.1
matplotlib 3.3.4
seaborn 0.11.1

Firefox 90.0.1

macOS Big Sur (11.4)

I'm also posting this on the VS Code - Jupyter Extension and JupyterLab forums/github.

[rchiodo]

Internally those events in VS Code are fired by this code here:
https://github.com/jupyter-widgets/ipywidgets/blob/355eea58b154e9857085e0a37a989174a6b68786/jupyterlab_widgets/src/manager.ts#L70

Which I believe happens when the kernel object is not being listened to in the UI script. Not sure why it wouldn't be though.

[DonJayamanne]

@mwh3780 Apologies for never getting to this, we're looking at fixing this issue in the current iteration.
Marking as a duplicate of #9503 (which will be fixed in the next release)