support for jupyterlite + xeus_python kernel

[stevejpurves]

Context

Enable support for using kernels other than pyolite, starting with xeus-python

Proposal

No response

Tasks and updates

No response

[stevejpurves]

Idea:

• can the xeus-python build process be used independently of the full jupyterlite cli build process, then this could be invoked to prepare assets that can be shipped with thebe-lite https://github.com/jupyterlite/xeus-python-kernel/blob/main/jupyterlite_xeus_python/env_build_addon.py#L122

[stevejpurves]

cc @martinRenou

[martinRenou]

can the xeus-python build process be used independently of the full jupyterlite cli build process, then this could be invoked to prepare assets that can be shipped with thebe-lite

This cannot currently be done, but we can rewrite xeus-python-kernel's build logic in a way that make it easier to use from outside.

I'll have a look at this this afternoon or tomorrow.

[martinRenou]

can the xeus-python build process be used independently of the full jupyterlite cli build process

In this PR I've been working on splitting up the JupyterLite addon in two parts:

• the emscripten env build and packing part
• the addon itself

This way installing jupyterlite-xeus-python will provide an command jupyterlite-xeus-python-build that you can use for packing the env:

> jupyterlite-xeus-python-build --help
                                                                                                                                                                                                                                                                                                                                                            
 Usage: jupyterlite-xeus-python-build [OPTIONS]                                                                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                                                                            
 Build and pack an emscripten environment.                                                                                                                                                                                                                                                                                                                  
                                                                                                                                                                                                                                                                                                                                                            
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --python-version                              TEXT  [default: 3.10]                                                                                                                                                                                                                                                                                      │
│ --xeus-python-version                         TEXT                                                                                                                                                                                                                                                                                                       │
│ --packages                                    TEXT  The list of packages you want to install                                                                                                                                                                                                                                                             │
│ --environment-file                            TEXT  The path to the environment.yml file you want to use                                                                                                                                                                                                                                                 │
│ --root-prefix                                 TEXT  [default: /tmp/xeus-python-kernel]                                                                                                                                                                                                                                                                   │
│ --env-name                                    TEXT  [default: xeus-python-kernel]                                                                                                                                                                                                                                                                        │
│ --empack-config                               TEXT  The empack config file to use. If not provided, the default empack config will be used                                                                                                                                                                                                               │
│ --output-path                                 TEXT  The directory where to output the packed environment [default: .]                                                                                                                                                                                                                                    │
│ --build-worker           --no-build-worker          Whether or not to build the TypeScript worker code for using xeus-python in JupyterLite [default: no-build-worker]                                                                                                                                                                                   │
│ --help                                              Show this message and exit.                                                                                                                                                                                                                                                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

It's not released yet and it's very much experimental, I'm not sure this API should stay as is, but that could probably help for using in thebe.

Also thebe could use the Python API directly instead of the command line if needed.

[psychemedia]

I note that there is also now an experimental jupyterlite R kernel: https://github.com/r-wasm/jupyterlite-webr-kernel

It would be good to be able to run R backed Jupyter Books too...

[michaelweinold]

@stevejpurves / @martinRenou, do you have an update on the xeus-python integration with thebe-lite?

From what I can see at docs-core/5-jupyterlite.md and packages/lite/README.md, this is not currently possible?

If there are specific issues that would need to be resolved first, we would be happy to dedicate some resources.

[stevejpurves]

@michaelweinold thanks for the ping! There are a few outstanding issues with the juptyerlite integration here which I think will be largely fixed by changes here #606 -- i've started tackling this and should have a PR posted this week -- once we're able to pick up the latest pyolite wheels independently from jupyterlite, that simplified deployment and might be a good starting point for looking at how to load xeus and other kernels.

@martinRenou i don't know if the xeus kernel is also planning-on/is shipping it's wheels independently as jupyter-pyodide-kernel has?

[martinRenou]

@martinRenou i don't know if the xeus kernel is also planning-on/is shipping it's wheels independently as jupyter-pyodide-kernel has?

I'm not sure I understand the question? Are you talking about the wheel for the jupyterlite-xeus-python package, or wheels for installed packages in the emscripten environment?

[stevejpurves]

The wheel for jupyterlite-xeus-python...
... but I see that it is already on npm here https://www.npmjs.com/package/@jupyterlite/xeus-python-kernel 👍🏼

[martinRenou]

The wheel for jupyterlite-xeus-python is on PyPi: https://pypi.org/project/jupyterlite-xeus-python.

The NPM package is only if you want to depend on it in a front-end extension, which could be the case for thebe indeed. Though it may be nicer for thebe to be able to load federated labextensions instead of hardcoding NPM dependencies?

[stevejpurves]

yes - i think we should move directly to loading the federated lab extensions as (as far as I understand t) then we can support multiple kernels with one pattern and also have these loaded from elsewhere.

• https://webpack.js.org/concepts/module-federation/
• https://webpack.js.org/concepts/module-federation/#dynamic-remote-containers