JupyterLab 3 update

[martinRenou]

This is the quick path to get this working with JupyterLab 3, not finished yet. I still need to make jupyterlab-lsp a federated labextension.

[bollwyvl]

if these end up in the dist (which they might) we will want them formatted...

[bollwyvl]

Please bring back python -m pip install -e . --ignore-installed --no-deps -vv... it's not too livehackable, today (vs #278), but would like to maintain that experience.

[martinRenou]

Sure, I'll bring it back

[martinRenou]

Although -e will not install the data_files, but we need them installed

[bollwyvl]

as has been mentioned elsewhere: maybe we have the structure in-package already looks like the system-of-interest, e.g. etc/ and share/ so the patterns are more concise.

[bollwyvl]

we need to add checking all of these (and probably other things) to integrity.py

[bollwyvl]

wut

[martinRenou]

This makes it importable :) enabling jupyter labextension develop

[bollwyvl]

distributed package name != import name... this is explicitly different because of that. like i mentioned elsewhere, if something in the stack is conflating the two, it needs to be nuked from orbit. If it has to be like this, then that amplifies the need for it to be a standalone package.

[martinRenou]

I am totally fine reverting this change, just note that you won't be able to use jupyter labextension develop.

then that amplifies the need for it to be a standalone package

Yes, although I wanted to keep this PR simple for now and not make another package.

[jasongrout]

See also jupyterlab/jupyterlab#9576

[bollwyvl]

Pretty sure we discussed this, but think we might want to have a dependency tree like:

jupyterlab_lsp ---+---> jupyter_lsp ----+---> jupyter_server
                  \---> jupyterlab ----/

Motivation: we have been trying really hard to make the contract between the frontend and backend packages "regular" (e.g. schema-constrained). If the frontend starts getting shipped with the backend, this would become less formal, and downstreams that wish to extend the functionality would be at a disadvantage, and more likely to fork.

Further, if we split out and version individual features in the future, jupyterlab_lsp can become a dependency-only metapackage, we can add 100 packages, and jupyter_lsp (but more importanty, docs) wouldn't change at all.

We'll do a major version bump for jupyter_lsp, and a can copy that version for jupyterlab_lsp, and they'll have pretty hard pins between the two of them. If anything, I've considered having a third package which only held "our" JSON schema version of the LSP and the definition of the extra metadata stuff we need... but that's for another day.

[martinRenou]

Pretty sure we discussed this, but think we might want to have a dependency tree like

Not sure I understand the tree structure you are drawing here.

But I guess you are mentioning to make a separate Python package for the labextension? It might be indeed nicer. Although I didn't want this PR to drastically change the code structure.

The PR works now. You should get it to work with:

jlpm bootstrap
pip install .
jupyter serverextension enable --sys-prefix --py jupyter_lsp

Please don't hesitate to push to my branch if you want to change/polish some things.

[bollwyvl]

Yeah, data_files (however it's wrapped) is gross, for _precisely_ this reason. Would running "labextension develop" do the thing? I'll be digging into what is happening with labextension develop, but for reference, I use postBuild as part of my local workflow, to ensure it keeps working without building on binder... So I'm very motivated to have binder end up with a ready-to-debug environment for someone's pr... We should probably have a separate branch for binder that's made of our up-to-version info without any development junk (but still node, gah), but that, too, is for another day...

[martinRenou]

Would running "labextension develop" do the thing?

Yes, it should. labextension develop will make a symlink from the local build to the share/jupyter/labextension directory. But we would need to figure out the package name issue (jupyter-lsp vs jupyter_lsp). This is something that should be fixed in JupyterLab itself maybe, it tries to import the package to find the _jupyter_labextension_paths function. Maybe JupyterLab could use importlib.importmodule('jupyter-lsp') so that it works.

[bollwyvl]

Welp, we need to make a new branch for you to target, in the first place... We have users on lab 2 will need to support for the foreseeable future. But have no fear of making drastic changes if they make sense for (roughly in order): - end users - volunteer maintainer - first time contributors - downstream developers - jupyterhub administrators - downstream package maintainers Regarding the volunteer bit: it'll probably be tomorrow before I can get back on this!

[karlaspuldaro]

Nice work! Looking forward to seeing this in lab3 :)
A couple suggestions so far.

As per comments in #411 maybe this PR can update the docs to use node 12+ required to dev for lab3.
And I saw rc10 is out, that could also be tested out and updated here.

If the suggestions are valid, let me know if you need any help then I can create a PR for your branch :)

[martinRenou]

Thanks for the suggestions. Yes, don't hesitate to open a PR to my branch :)

[bollwyvl]

This seems close to coming home... @krassowski et al: how do we want to handle landing this? Two options:

• create a 3.x, and land this PR against it
• create a 2.x, and land this against master

As there is still no firm release date for lab 3, i'm favoring the former, but it's kinda all the same 🤷

Also as this is going to be turbulent, we might want to investigate looking into the more complicated RTD setup which gives us versioned releases.

[martinRenou]

As pointed out by @karlaspuldaro only the readthedocs build is triggered on the CI. Any idea why Github actions is not triggered here? Maybe it's because of the draft status, I will change this

[martinRenou]

I will need to rebase and resolve conflicts

[krassowski]

Yes, I think it was the draft status, but if CI does not trigger with the next push let me know and I will investigate further. I will look into branching/catching up with releases later on Wednesday.

[martinRenou]

🎉 Thanks a lot!