Distribute the extensions as Python packages

[jtpio]

Fixes #235

This takes the approach of packaging all the extension at once into a jupyterlab-renderers package.

~So a pip install jupyterlab-renderers will install all the extensions.`

Edit: Each extension is distributed as its own Python package.

JupyterLab v3.0.9
/home/username/miniforge3/envs/jupyter-renderers/share/jupyter/labextensions
        @jupyterlab/katex-extension v3.1.0 enabled OK (python, jupyterlab-renderers)
        @jupyterlab/vega2-extension v3.0.0 enabled OK (python, jupyterlab-renderers)
        @jupyterlab/mathjax3-extension v4.1.1 enabled OK (python, jupyterlab-renderers)
        @jupyterlab/fasta-extension v3.0.0 enabled OK (python, jupyterlab-renderers)
        @jupyterlab/geojson-extension v3.0.0 enabled OK (python, jupyterlab-renderers)
        @jupyterlab/vega3-extension v3.0.0 enabled OK (python, jupyterlab-renderers)

TODO

• Start with @jupyterlab/fasta-extension
• Package the other extensions
• Check if the jupyter labextension develop script can be used with multiple packages and this setup
• Rebase once Switch to GitHub Actions on CI #242 is merged to pick up the GitHub Actions workflow and build from it
• Update CI workflow to install via pip
• Update dev setup and READMEs
• Update the individual READMEs

[jtpio]

Check if the jupyter labextension develop script can be used with multiple packages and this setup

The develop script might need another pass to support the setup of 1 Python package for multiple lab extensions:

$ jupyter labextension develop . --overwrite
Installing /path/to/repo/jupyter-renderers/jupyterlab-renderers/labextension/@jupyterlab/fasta-extension -> @jupyterlab/fasta-extension
Building extension in /path/to/repo/jupyter-renderers/jupyterlab-renderers
An error occured.
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/repo/jupyter-renderers/jupyterlab-renderers/package.json'
See the log file for details:  /tmp/jupyterlab-debug-mn3dgl19.log

[jasongrout]

Some of these are alternatives that don't make sense to install together. For example, it doesn't make sense to install both mathjax 3 and katex as they stomp on each other to be the builtin math renderer (and disable the builtin mathjax 2 renderer).

I was thinking about this when upgrading to jlab 3, and how we make prebuilt extensions. I think we can either have multiple python packages in this repo, or split things out to multiple repos (which I think would make updates and maintenance onerous). I would suggest multiple python packages in this repo over splitting things out into multiple repos, at least as a first experiment.

[jtpio]

I would suggest multiple python packages in this repo over splitting things out into multiple repos, at least as a first experiment.

Right. This sounds like we will end up with a "Python monorepo", so this might add some complexity especially if we want to keep some files like setup.py in sync (similar for all the extensions). But maybe there is a way to streamline the setup so it's convenient to work with.

Agree it's still better than maintaining multiple repos 👍

[jtpio]

Python monorepo

The closest we have to this for now is probably the Extension Examples repo: https://github.com/jupyterlab/extension-examples

[jtpio]

So the repo is now a collection of Python packages.

Each of the extensions was updated using the upgrade script:

python -m jupyterlab.upgrade_extension .

Check if the jupyter labextension develop script can be used with multiple packages and this setup

The develop script can be run manually in each folder now.

[jtpio]

skipLibCheck wasn't originally used, but updating the dependencies led to TS compile errors for what looks like conflict with the underlying vega-lite dependency:

https://github.com/jupyterlab/jupyter-renderers/runs/2033195852

node_modules/vega-lite/build/src/compile/selection/selection.d.ts(11,10): error TS2440: Import declaration conflicts with local declaration of 'SelectionComponent'.
  node_modules/vega-lite/build/src/log.d.ts(23,5): error TS2416: Property 'level' in type 'LocalLogger' is not assignable to the same property in base type 'LoggerInterface'.
    Type '() => this' is not assignable to type '{ (_: number): this; (): number; }'.
      Type 'this' is not assignable to type 'number'.
        Type 'LocalLogger' is not assignable to type 'number'.

So leaving skipLibCheck: true to make it simpler for now since Vega 3 is deprecated.

[jtpio]

So the repo is now a collection of Python packages.

Keeping the packages folder structure and the lerna setup for now. This makes it possible to use the previous monorepo workflow for example to install dependencies with jlpm and build everything at once with jlpm run build.

[blink1073]

Thanks! Needs a rebase for the yarn lock.

[jasongrout]

Thanks!

[jasongrout]

I've published these python packages, and added you, @jtpio as a publisher as well on pypi.

[jasongrout]

I guess next up is making conda-forge feedstocks, if we want to go that far?

[jtpio]

Perfect, thanks!

I guess next up is making conda-forge feedstocks, if we want to go that far?

Probably yes. Maybe it's possible to generate most of the recipes with grayskull and submit a single PR to add them all at once.

[jtpio]

Opened conda-forge/staged-recipes#14245. Recipes were generated with:

grayskull pypi jupyterlab-fasta jupyterlab-geojson jupyterlab-katex jupyterlab-mathjax3 jupyterlab-vega2 jupyterlab-vega3