Enabling rendering on nbviewer by persisting state of anywidget in Jupyter Notebook

[andersy005]

@manzt and i have been experimenting with anywidget + @carbonplan/maps in https://github.com/manzt/carbonplan. The notebook works perfectly fine when connected to a live kernel. however, i am facing an issue with persisting the widget's state within the notebook and having external services like nbviewer render the initial state of the widget.

to illustrate the problem, i have shared a sample notebook at https://nbviewer.org/gist/andersy005/f8bca6c542135a75ab3e11203eada3a1. as you can observe, the last cell of the notebook is not being rendered.

any guidance on how to resolve this issue?

Screenshare.-.2023-10-12.4_24_14.PM.mp4

Cc @katamartin

[rgbkrk]

I'm a bit new to how widgets persist state in the notebook so for my own sake (and maybe others) I'm going to document what I see in the notebook. I can see the widget model in the output:

There is no other reference to the widget model in the source of the notebook and nothing for nbviewer to render (other than that text/plain entry).

When I try the simplest widget in JupyterLab, the IntSlider I'm also not seeing the persistence.

Cell:

from ipywidgets import IntSlider
islide = IntSlider()
islide

Notebook:

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "088fffff-0cb1-4055-a722-2635371d3be1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9480df1fa4f04356873a4969ef4038c4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "IntSlider(value=0)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from ipywidgets import IntSlider\n",
    "islide = IntSlider()\n",
    "islide"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

If I recall correctly, the classic notebook had a "Save Widget State" button. I don't have that in JupyterLab.

[rgbkrk]

It looks like there's a setting that has to be enabled in JupyterLab to automatically save widget state

After doing that and also making sure to update my jupyterlab and ipywidgets installation, I restarted JupyterLab and see a big pile of state:

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "088fffff-0cb1-4055-a722-2635371d3be1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "03a522b9129949d8ae7a802b0f344980",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "IntSlider(value=0)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from ipywidgets import IntSlider\n",
    "islide = IntSlider()\n",
    "islide"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "03a522b9129949d8ae7a802b0f344980": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "IntSliderModel",
      "state": {
       "behavior": "drag-tap",
       "layout": "IPY_MODEL_1db186f70ff646aea16422745dea7a8b",
       "style": "IPY_MODEL_30cb35a482a54c17a2173a45376f43ea",
       "value": 85
      }
     },
     "1db186f70ff646aea16422745dea7a8b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "30cb35a482a54c17a2173a45376f43ea": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "SliderStyleModel",
      "state": {
       "description_width": ""
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

However, that doesn't render in notebook viewer. https://nbviewer.org/gist/rgbkrk/d11992979381414b32d18b16be2d64cf

I've been away from working on nbviewer and nbconvert so I don't know how it takes the widget state into account.

[manzt]

However, that doesn't render in notebook viewer. nbviewer.org/gist/rgbkrk/d11992979381414b32d18b16be2d64cf

I'm not sure how nbviewer is implemented, but nbconvert works for rendering widgets to HTML. flekschas/jupyter-scatter#81 (comment)

If you nbconvert --execute the notebook, it will embed the widget model state, otherwise you'll need to make sure the model state is embedded (i.e., "Save Widget State Automatically" in JupyterLab).

For clarity, @andersy005 this is not an issue with anywidget, but Jupyter Widgets (standard ipywidgets do not render in nbviewer either, thanks to @rgbkrk example). But thank you @rgbkrk for the exploration, and maybe we can at least figure out if this is unexpected or expected behavior!

[andersy005]

@rgbkrk / @manzt, thank you for looking into this and suggesting workarounds. i was able to convert the executed notebook to HTML and then served it from an s3 bucket: https://carbonplan-share.s3.us-west-2.amazonaws.com/leap-demo.html

and it appears to be working: initial states of the widgets are captured as expected

[manzt]

Awesome! @andersy005 if you use ipywidgets.jslink instead of ipywidgets.link you can also link the inputs on the client side (i.e., sliders and dropdowns will still work in the HTML-only version).

import ipywidgets

colormap = ipywidgets.Dropdown(options=["warm", "fire", "water"])
clim = ipywidgets.FloatRangeSlider(min=-20, max=30)
opacity = ipywidgets.FloatSlider(min=0, max=1, step=0.001)
region = ipywidgets.Checkbox(description="Region")

-- ipywidgets.link((map_widget, "colormap"), (colormap, "value"))
-- ipywidgets.link((map_widget, "clim"), (clim, "value"))
-- ipywidgets.link((map_widget, "opacity"), (opacity, "value"))
-- ipywidgets.link((map_widget, "region"), (region, "value"))
++ ipywidgets.jslink((map_widget, "colormap"), (colormap, "value"))
++ ipywidgets.jslink((map_widget, "clim"), (clim, "value"))
++ ipywidgets.jslink((map_widget, "opacity"), (opacity, "value"))
++ ipywidgets.jslink((map_widget, "region"), (region, "value"))

ipywidgets.VBox([
    ipywidgets.HBox([colormap, opacity, clim]),
    region,
    map_widget,
])

[andersy005]

thank you very much, @manzt! the jslink changes work :) with the exception of any selection widgets (dropdown, radiobuttons, select, etc), and this appears to be a well known issue upstream:

• jslink not working for dropdown widget, why?  jupyter-widgets/ipywidgets#2382

however, i'm satisfied with dropping the dropdown widget for the time being: https://carbonplan-share.s3.us-west-2.amazonaws.com/leap-demo.html

[manzt]

Great! I'm going to mark this issue as resolved!

[manzt]

Ah, actually realizing this doesn't address that things don't seem to be working in nbviewer. Still need to look into that, but it doesn't seem like any widgets render with nbviewer.

[manzt]

Seems related:

• nbviewer not working with Interact/ipywidgets (sliders, dropdowns ++) jupyter/nbviewer#987

[kdheepak]

I'm not sure if this is nbviewer only, because I'm also seeing the same issue with quarto:

This is what it is supposed to look like:

Also, I'm not seeing an issue when rendering with folium and quarto but I am seeing the issue rendering with lonboard and quarto.

Here's the notebook that you can use to reproduce this issue:

https://github.com/user-attachments/files/17976051/Archive.zip

Related issue on Quarto that I will probably close:

quarto-dev/quarto-cli#11594

[manzt]

@kdheepak this is pretty perplexing to me. I ran the notebook for myself locally and yes, it doesn't seem to work with quarto. However, the logs say that first it is looking for anywidget v0.9.* and then failing on anywidget v2.0.0.

I also am seeing different behavior in different browsers. My best guess is that this is somehow coming from quarto and how it has set up its HTML widget manager. I'm able to extract the widget metadata from the generated index.html, so the state looks like it is all there:

cat index.html | \
  sed -n '/<script type="application\/vnd.jupyter.widget-state+json">/,/<\/script>/p' | \
  sed '1d;$d' | \
  jq '.state |= map_values(del(.state))'
# {
#   "state": {
#     "0418f9b758f04d7889cfc9f4332f42b9": {
#       "model_module": "anywidget",
#       "model_module_version": "~0.9.*",
#       "model_name": "AnyModel"
#     },
#     "cacac92874f74116b4888983a3a056ac": {
#       "model_module": "@jupyter-widgets/base",
#       "model_module_version": "2.0.0",
#       "model_name": "WidgetModel"
#     },
#     "caf9424537bd43af875312b056669671": {
#       "model_module": "@jupyter-widgets/base",
#       "model_module_version": "2.0.0",
#       "model_name": "LayoutModel"
#     },
#     "e7509faff7254447995e244fd217a612": {
#       "model_module": "@jupyter-widgets/base",
#       "model_module_version": "2.0.0",
#       "model_name": "WidgetModel"
#     }
#   },
#   "version_major": 2,
#   "version_minor": 0
# }

I don't think this is an issue with anywidget because the same notebook works rendering to HTML with nbconvert:

uv run jupyter nbconvert --to=html --execute index.ipynb