Binder does not work with Jupyter Server v2

[fcollonval]

Description

If you install jupyter-server v2 as a dependency on Binder, binder will not work.

Reproduce

You could reproduce the error locally by

1. installing jupyter-server v2, jupyterlab and notebook
2. start JupyterLab with: jupyter notebook --NotebookApp.default_url="/lab"
3. You are stuck on the login page

I try setting password and token to blank or to something. But it does not work. There is something about the cookie that gets cleaned I guess.

Expected behavior

You can access the server.

Context

• Operating System and version: debian 11
• Browser and version: firefox 98
• Jupyter Server version: v2rc3

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here, if applicable.

[fcollonval]

If I set a password (the same but it does not matter) for notebook and jupyter-server, here is the server log:

[I 12:23:07.559 NotebookApp] 302 POST /login?next=%2Flab (127.0.0.1) 21.400000ms
[D 12:23:07.564 NotebookApp] Error unpacking user from cookie: cookie=5690159cc57d4fa8940989db1d51e2bd
    Traceback (most recent call last):
      File "/home/fcollonval/miniconda3/envs/lab36/lib/python3.10/site-packages/jupyter_server/auth/identity.py", line 401, in get_user_cookie
        return self.user_from_cookie(user_cookie)
      File "/home/fcollonval/miniconda3/envs/lab36/lib/python3.10/site-packages/jupyter_server/auth/identity.py", line 308, in user_from_cookie
        user = json.loads(cookie_value)
      File "/home/fcollonval/miniconda3/envs/lab36/lib/python3.10/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
      File "/home/fcollonval/miniconda3/envs/lab36/lib/python3.10/json/decoder.py", line 340, in decode
        raise JSONDecodeError("Extra data", s, end)
    json.decoder.JSONDecodeError: Extra data: line 1 column 8 (char 7)
[E 12:23:07.564 NotebookApp] Error unpacking user from cookie: Extra data: line 1 column 8 (char 7)
[W 12:23:07.564 NotebookApp] Clearing invalid/expired login cookie username-localhost-8888
[I 12:23:07.565 NotebookApp] 302 GET /lab (127.0.0.1) 1.020000ms
[D 12:23:07.567 NotebookApp] Using contents: services/contents
[D 12:23:07.568 NotebookApp] 200 GET /login?next=%2Flab (127.0.0.1) 1.110000ms

[vidartf]

Is this still an issue?

[fcollonval]

As far as I know, yes. The easiest setup to make binder run with Jupyter Server rather than the old Notebook server is to add a start file in the binder configuration with the following content:

#!/bin/bash

set -e

echo $@

exec jupyter-lab "${@:4}"

[fcollonval]

I tried to dig a bit more into this. But this is really tricky. It does not work locally with a token neither if you do the following:

conda create -n testlab jupyterlab=3
conda list
jupyter notebook --NotebookApp.default_url="/lab"

You should have JupyterLab 3.5 or later, jupyter-server 2 and notebook 6.

You will face the login page. The trouble is definitely a incompatibility between the new identity provider and the previous way of dealing with authentication. When going to /, notebook redirects to /lab. Then the identity provider redirect to /login?next=/lab using nbclassic.

For now in JupyterLab, I implemented a test when loading JupyterLab as old notebook extension to log a message if jupyter server is version 2. It stops the process if the default url is /lab and otherwise skip adding JupyterLab as extension.

Xref jupyterlab/jupyterlab#13716

[matthewfeickert]

As far as I know, yes. The easiest setup to make binder run with Jupyter Server rather than the old Notebook server is to add a start file in the binder configuration with the following content:

#!/bin/bash

set -e

echo $@

exec jupyter-lab "${@:4}"

In the context of jupyterhub/repo2docker#1231, do you have any advice on how to try to force this behavior from a binder/Dockerfile's ENTRYPOINT? It might not be possible as I assume repo2docker is going to attempt to override things and expect that the Docker image's ENTRYPOINT is compatible.

[JasonWeill]

As mentioned in jupyter/notebook#6702 linked above, this may also affect Notebook's ability to run with Jupyter Server v2 (when launched directly, not when launched from within JupyterLab).

[takluyver]

Not only Binder - I'm also seeing this running jupyter notebook locally.

[Zsailer]

I've got a PR attempting to fix this, #1221.

The combination of "classic" notebook server (handlers inherit from IPythonHandler) running "modern" jupyter_server extensions (handlers that inherit from JS 2.0 JupyterHandler) is broken.

This is because the modern jupyter_server extensions—which create an identity provider on the fly—overwrites the login cookie set by the classic handlers because they use the same cookie name.

[bollwyvl]

Guess this can probably be closed at this point?