repo2docker fails to launch into jupyter lab properly from valid minimal Dockerfile

[matthewfeickert]

Bug description

When repo2docker 2022.10.0 builds a valid minimal Dockerfile and then attempts to launch into it with (I think?)

docker run -it --rm -p 8888:8888 my-image jupyter notebook --NotebookApp.default_url=/lab/ --ip=0.0.0.0 --port=8888

it fails to enter into the launcher environment and is blocked by a token authentication screen that is not bypassable with a valid token.

Expected behaviour

Launch into a Jupyter Lab environment similar to repo2docker having run

docker run -it --rm -p 8888:8888 my-image jupyter lab --ip=0.0.0.0 --port=8888

Actual behaviour

When launching the environment from the URL with the token presented the user is brought to a login screen shown below that notes "Token authentication is enabled".

If the user tries to log in with the token in the URL they are not able to.

How to reproduce

This repo provides a full minimal failing example: https://github.com/matthewfeickert/failing-repo2docker-with-dockerfile

The following is my writeup pasted from the MFE repo:

Minimal Failing Example:

Minimal Failing Example

The Dockerfile in this repository under binder/ is a modified version of the example given in the binder-examples/minimal-dockerfile project.
The modifications are minor in that it adopts python:3.10-slim-bullseye as the base image and installs all Python packages in a NB_USER controlled virtual environment which is automatically activated as its bin/ directory is on $PATH.
The rest of it follows the Preparing your Dockerfile section of the Use a Dockerfile for your Binder repository tutorial website.
The Dockerfile builds successfully when repo2docker is run locally on the repository

# if you don't use pipx just pip install in your local virtual environment
$ pipx install jupyter-repo2docker
$ repo2docker --version
2022.10.0
$ repo2docker .

However, when launching the environment from the URL with the token presented the user is brought to a login screen shown below that notes "Token authentication is enabled".

If the user tries to log in with the token in the URL they are not able to.

From the tutorial it seems that repo2docker is running the equivalent of

docker run -it --rm -p 8888:8888 my-image jupyter notebook --NotebookApp.default_url=/lab/ --ip=0.0.0.0 --port=8888

after it builds the image. For simplicity of naming, if we retag the image

docker tag <r2d-hash>:latest matthewfeickert/failing-repo2docker-with-dockerfile:latest

(or just build the image ourselves

docker build -f binder/Dockerfile -t matthewfeickert/failing-repo2docker-with-dockerfile:latest .

)

and then run it as repo2docker is

docker run -it --rm -p 8888:8888 matthewfeickert/failing-repo2docker-with-dockerfile:latest jupyter notebook --NotebookApp.default_url=/lab/ --ip=0.0.0.0 --port=8888

we get the same issue.
It seems the problems stems from --NotebookApp.default_url=/lab/ as if we run without this then we can run the classic Jupyter notebook environment fine

docker run -it --rm -p 8888:8888 matthewfeickert/failing-repo2docker-with-dockerfile:latest jupyter notebook --ip=0.0.0.0 --port=8888

and if we run with jupyter lab things work fine

docker run -it --rm -p 8888:8888 matthewfeickert/failing-repo2docker-with-dockerfile:latest jupyter lab --ip=0.0.0.0 --port=8888

I understand from the tutorial that the --NotebookApp.default_url=/lab/ is required for Binder to act as a safeguard as

If you install the classic notebook interface but not JupyterLab, you must manually change your mybinder.org URLs from /lab to /tree as described in the user interface documentation.
Otherwise, you might get a 404: Not Found error when launching your project on binder.

but it seems there is some interaction problem here for repo2docker.

this is also cross-posted with binder-examples/minimal-dockerfile#8 (comment).

Your personal set up

• OS: [e.g. linux, OSX] Ubuntu 22.04 LTS
• Docker version: docker version

Client: Docker Engine - Community
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:04 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.9
  Git commit:       42c8b31
  Built:            Thu Dec 15 22:25:49 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• repo2docker version: repo2docker --version

2022.10.0

[matthewfeickert]

I don't understand things yet, but it seems that there is some interaction effect between jupyter-server v2.0+ and other requirements as simply just adding jupyter-server<2.0.0' is able to get repo2docker working.

 # Install necessary jupyter dependencies
 # Install jupyterhub to allow use of the repository with an authenticated Binder
 RUN python -m pip --no-cache-dir install --upgrade pip setuptools wheel && \
-    python -m pip --no-cache-dir install jupyter jupyterlab jupyterhub && \
+    python -m pip --no-cache-dir install jupyter jupyterlab jupyterhub 'jupyter-server<2.0.0' && \
     command -v python && \
     python -m pip list

I'm poking on this over at matthewfeickert/failing-repo2docker-with-dockerfile#4.

[matthewfeickert]

I have failing and passing pip-tools lock files posted in matthewfeickert/failing-repo2docker-with-dockerfile#4 (comment) for reproducibility. Without being knowledgeable about the differences between them it looks like the main difference is just in jupyter-server.

[manics]

This sounds like an upstream problem jupyter-server/jupyter_server#1038