[ucmerced] Issues with voila in 2i2c-hubs-image with Jijnja2 >= 3.1.x

[pnasrat]

Context

1. Log in to hub
2. Create an R notebook
3. enter print("Hello, World!) in cell and run
4. click on Render With Voila

Result: 404 when rendering with voila - For the Voila error Dev Tools gives a 404 for below url:

https://ucmerced.2i2c.cloud/user/pnasrat@2i2c.org/voila/render/Untitled1.ipynb

Voila tracebacks: https://cloudlogging.app.goo.gl/2HzJPBaNbijC4qAd7

"[W 2023-03-15 18:32:35.052 SingleUserLabApp manager:332] voila.server_extension | error adding extension (enabled: True): The module 'voila.server_extension' could not be found (cannot import name 'contextfilter' from 'jinja2' (/opt/conda/lib/python3.7/site-packages/jinja2/__init__.py)). Are you sure the extension is installed?"

For PDF rendering

Proposal

No response

Updates and actions

No response

[pnasrat]

Jinja2 is 3.1.2 in image this issue looks related d2l-ai/d2l-book#46

quay.io/2i2c/2i2c-hubs-image:05c6a27dc942

docker pulled that image and ran

[W 2023-03-15 19:05:50.405 SingleUserLabApp manager:332] voila.server_extension | error adding extension (enabled: True): The module 'voila.server_extension' could not be found (cannot import name 'contextfilter' from 'jinja2' (/opt/conda/lib/python3.7/site-packages/jinja2/__init__.py)). Are you sure the extension is installed?
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.7/site-packages/jupyter_server/extension/manager.py", line 175, in _validate_name
        self._module, self._metadata = get_metadata(name)
      File "/opt/conda/lib/python3.7/site-packages/jupyter_server/extension/utils.py", line 60, in get_metadata
        module = importlib.import_module(package_name)
      File "/opt/conda/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 728, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/opt/conda/lib/python3.7/site-packages/voila/__init__.py", line 11, in <module>
        from .server_extension import _load_jupyter_server_extension # noqa
      File "/opt/conda/lib/python3.7/site-packages/voila/server_extension.py", line 19, in <module>
        from .handler import VoilaHandler
      File "/opt/conda/lib/python3.7/site-packages/voila/handler.py", line 29, in <module>
        from .exporter import VoilaExporter
      File "/opt/conda/lib/python3.7/site-packages/voila/exporter.py", line 15, in <module>
        from jinja2 import contextfilter
    ImportError: cannot import name 'contextfilter' from 'jinja2' (/opt/conda/lib/python3.7/site-packages/jinja2/__init__.py)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.7/site-packages/jupyter_server/extension/manager.py", line 320, in add_extension
        extpkg = ExtensionPackage(name=extension_name, enabled=enabled)
      File "/opt/conda/lib/python3.7/site-packages/jupyter_server/extension/manager.py", line 166, in __init__
        super().__init__(*args, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 1361, in __init__
        value = self._traits[key]._cross_validate(self, getattr(self, key))
      File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 746, in _cross_validate
        value = obj._trait_validators[self.name](obj, proposal)
      File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 1233, in __call__
        return self.func(*args, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/jupyter_server/extension/manager.py", line 179, in _validate_name
        "sure the extension is installed?".format(name=name, e=e)
    jupyter_server.extension.utils.ExtensionModuleNotFound: The module 'voila.server_extension' could not be found (cannot import name 'contextfilter' from 'jinja2' (/opt/conda/lib/python3.7/site-packages/jinja2/__init__.py)). Are you sure the extension is installed?
[I 2023-03-15 19:05:50.405 SingleUserLabApp manager:344] notebook_shim | extension was successfully linked.
JUPYTERHUB_API_TOKEN env is required to run jupyterhub-singleuser. Did you launch it manually?

jovyan@6028d2964522:~$ pip install --upgrade Jinja2==3.0.3
Collecting Jinja2==3.0.3
  Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 kB 4.0 MB/s eta 0:00:00
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.7/site-packages (from Jinja2==3.0.3) (2.1.2)
Installing collected packages: Jinja2
  Attempting uninstall: Jinja2
    Found existing installation: Jinja2 3.1.2
    Uninstalling Jinja2-3.1.2:
      Successfully uninstalled Jinja2-3.1.2
Successfully installed Jinja2-3.0.3

[notice] A new release of pip is available: 23.0 -> 23.0.1
[notice] To update, run: pip install --upgrade pip
jovyan@6028d2964522:~$ jupyterhub-singleuser 
[I 2023-03-15 19:07:17.089 SingleUserLabApp mixins:609] Starting jupyterhub single-user server version 3.1.1
[I 2023-03-15 19:07:17.089 SingleUserLabApp mixins:624] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.6.1
[I 2023-03-15 19:07:17.089 SingleUserLabApp mixins:624] Extending jupyter_server.serverapp.ServerApp from jupyter_server 1.23.6
[I 2023-03-15 19:07:17.314 SingleUserLabApp manager:344] jupyter_resource_usage | extension was successfully linked.
[I 2023-03-15 19:07:17.316 SingleUserLabApp manager:344] jupyter_server_fileid | extension was successfully linked.
[I 2023-03-15 19:07:17.316 SingleUserLabApp manager:344] jupyter_server_proxy | extension was successfully linked.
[I 2023-03-15 19:07:17.318 SingleUserLabApp manager:344] jupyter_server_ydoc | extension was successfully linked.
[I 2023-03-15 19:07:17.320 SingleUserLabApp manager:344] jupyterlab | extension was successfully linked.
[I 2023-03-15 19:07:17.320 SingleUserLabApp manager:344] jupytext | extension was successfully linked.
[W 2023-03-15 19:07:17.321 NotebookApp] 'terminado_settings' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2023-03-15 19:07:17.322 SingleUserLabApp manager:344] nbclassic | extension was successfully linked.
[I 2023-03-15 19:07:17.322 SingleUserLabApp manager:344] nbgitpuller | extension was successfully linked.
[I 2023-03-15 19:07:17.404 SingleUserLabApp nbserver:130] jupyter_tree_download | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server's extension paths.
[I 2023-03-15 19:07:17.404 SingleUserLabApp manager:344] jupyter_tree_download | extension was successfully linked.
[I 2023-03-15 19:07:17.404 SingleUserLabApp nbserver:130] jupyter_nbextensions_configurator | extension was found and enabled by notebook_shim. Consider moving the extension to Jupyter Server's extension paths.
[I 2023-03-15 19:07:17.404 SingleUserLabApp manager:344] jupyter_nbextensions_configurator | extension was successfully linked.
[I 2023-03-15 19:07:17.404 SingleUserLabApp manager:344] notebook_shim | extension was successfully linked.
[I 2023-03-15 19:07:17.404 SingleUserLabApp manager:344] voila.server_extension | extension was successfully linked.
JUPYTERHUB_API_TOKEN env is required to run jupyterhub-singleuser. Did you launch it manually?

[pnasrat]

I can confirm running jupyter notebook --ip 0.0.0.0 in port forwarded docker that rolling back Jinja2 to 3.0.x fixes voila

[pnasrat]

I was unable to replicated the PDF issue via jupyter-nbconvert Untitled.ipynb --to pdf --output foo.pdf the ucmerced hub, or locally docker running a notebook server (all worked for me)

[consideRatio]

Nice work!!! I wouldn't be surprised of voila has a fix for this, but hasnt fixed it in a version still supporting py3.7.

Seeing this image at python 3.7, that many packages no longer support, makes me think we must do or absolute best to avoid it for new hubs - this is the beginning of issues like this, not the end.

• Linking back to Intervention to have new hubs not couple to the 2i2c-hubs-image, but something more up to date #2336 as this to me was warning bell sounding loudly.

[pnasrat]

Fix in 2i2c-org/2i2c-hubs-image#8

[pnasrat]

Testing on 2i2c staging with quay.io/2i2c/2i2c-hubs-image:b16f88c26490 via configurator works where prior to image voila fails with 404 as above