[Proposal] New Github org for JupyterLab satellites

[blink1073]

In today's team meeting, we discussed the evolution of "core" repositories.

We decided that we needed a new org to meet the following goals

• Centralized place to work on JupyterLab related packages
• Uses the Jupyter contribution model and Code of Conduct
• Community-maintained - no guarantees about support from the core maintainers of JupyterLab

We notionally agreed to call this org jupyterlab-forge, as a nod to our friends at conda-forge.

The jupyterlab org would remain focused on JupyterLab itself and things that it depend on that we maintain, such as jupyterlab_server, and the apod example.

We should deprecate the jupyter-renderers and split it up into different repos on this new org.

Ideally, we should have a cookie cutter for new repos in this org and a guide to use them

[jasongrout]

I think we should also encourage people to host their own extensions, rather than this being a dumping ground for every jlab extension. In other words, this new org shouldn't convey any extra assurances to people about the extensions. I think if this new org does end up conveying some special status to people, we've failed in creating a robust extension community outside of JupyterLab proper.

I think there is a difference between this and conda-forge, in that conda-forge is very explicitly just a packaging org, as opposed to underlying package development in conda-forge. Also, conda-forge admins do explicitly help maintain all conda-forge packages (for example, mass migrations to new python versions or compiler versions). Are we volunteering to help shepherd packages to, say, jlab 2.0?

When would people create an extension here, as opposed to just hosting the extension in their own personal org or creating a new org?

[saulshanabrook]

When would people create an extension here, as opposed to just hosting the extension in their own personal org or creating a new org?

I believe just for visibility and as a way to reduce duplicate efforts. A single namespace can help with that, even by itself, without any guarantees of support.

I agree we would do less than conda forge does to keep things going, I think at most just being able to step in and add new people as maintainers if old people step away and new ones come.

[krassowski]

1. How does it compare to the jupyter-incubator (apart from being JupyterLab-specific)?
2. We would like to bring jupyterlab-lsp closer (https://github.com/krassowski/jupyterlab-lsp/issues/238) to foster collaboration (I think that some people may be more likely to contribute if it is not a personal repo). Would it be a good fit for us?
   • I would like to increase its visibility and enable others to pick up in case I can no longer maintain it
   • we think about having more than one repository at a time - would jupyterlab-forge allow for that, or would it be better served by a separate repo?

Please do not feel a need to comment on jupyterlab-lsp itself, just bringing my thoughts to you for an early feedback from the community side!

[jasongrout]

To clarify (and push back a bit to make sure we are addressing the underlying issues):

1. Are we implicitly (or explicitly?) encouraging people to develop their extensions in this new org? How much are we managing the org?
2. Are the projects all following the Jupyter code of conduct and have the Jupyter copyright and license?
3. Do these projects fall under the scope of the Jupyter governing bodies (the steering council, right now)?
4. What is the criteria for getting a project into the org?
5. What is the criteria for adding maintainers to projects in the org? Are we vetting them in any way?

I think my biggest concern is that this adds to our maintenance burden (managing an org), and it implicitly gives a stamp of approval on some community extensions, but not others.

Can we satisfy this need for publicity without having to consolidate development to a GitHub org? For example, a webpage that lists JupyterLab extensions (beyond the extension manager in jlab)?

If someone wants to move a project out of their personal github, creating a new github org is cheap. If someone is looking to pass on the project to others, instead of pushing that responsibility onto us in a contrib org, perhaps they should create an org and nominate others, or we should celebrate that OSS allows people to fork and carry on? Also if people are putting the projects there for others to take up, then it may turn out to be mostly a graveyard with an occasional exception.

[jasongrout]

@jcb91, @juhasch, @jfbercher - you have arguably the most successful contrib repo for Jupyter, the https://github.com/ipython-contrib/jupyter_contrib_nbextensions repo. What are your thoughts about having an org for third-party unofficial JupyterLab extensions? How much time does it take (has it taken) for you?

[jasongrout]

How does it compare to the jupyter-incubator (apart from being JupyterLab-specific)?

The Jupyter incubator is explicitly about projects that are being considered for adoption as official Jupyter projects, IIRC. This would be for community projects with a much lower bar.

[jasongrout]

It seems like these repos might be candidates for moving to a community extension org if we make one:

https://github.com/jupyterlab/jupyter-renderers
https://github.com/jupyterlab/jupyterlab_app
https://github.com/jupyterlab/jupyterlab_pygments
https://github.com/jupyterlab/jupyterlab-celltags
https://github.com/jupyterlab/jupyterlab-commenting
https://github.com/jupyterlab/jupyterlab-data-explorer
https://github.com/jupyterlab/jupyterlab-git
https://github.com/jupyterlab/jupyterlab-github
https://github.com/jupyterlab/jupyterlab-google-drive
https://github.com/jupyterlab/jupyterlab-hdf5
https://github.com/jupyterlab/jupyterlab-latex
https://github.com/jupyterlab/jupyterlab-metadata-service
https://github.com/jupyterlab/jupyterlab-monaco
https://github.com/jupyterlab/jupyterlab-mp4
https://github.com/jupyterlab/jupyterlab-shortcutui
https://github.com/jupyterlab/jupyterlab-telemetry
https://github.com/jupyterlab/jupyterlab-toc
https://github.com/jupyterlab/pull-requests

[telamonian]

@jasongrout That list looks good. Probably not jupyterlab-celltags though, since that one was actually pulled into core.

I'm not actually sure what we should do with the repo at https://github.com/jupyterlab/jupyterlab-celltags, which is badly out of date by now. It doesn't seem like great practice to just leave it lying around

[jasongrout]

Oh right. We can archive it, like we did for https://github.com/jupyterlab/jupyterlab-statusbar

[blink1073]

Thanks for the feedback @krassowski! I think it would be fine to have multiple related repos together in this org. We don't necessarily want an explosion of orgs, that would make it hard to find things.

[SylvainCorlay]

It seems like these repos might be candidates for moving to a community extension org if we make one

jupyterlab-pygments is a dependency of two core projects (nbconvert and Voilà). I think it belongs to core.

[blink1073]

Note: anything marked as core will also need to be added to the release process for JupyterLab.

[blink1073]

Basically we'll need to have integration checks for things like jupyterlab-pygments to make sure they work in a release.

[blink1073]

Ideally we'd have e2e checks in CI for that as well.

[blink1073]

There is a case to be made for taking jupyterlab-git into core, but I think it would also mean bringing nbdime into core since right now it is a dependency of jupyterlab-git and right now is not release-able by the core JupyterLab release managers (@jasongrout, @saulshanabrook, @afshin, and myself).

[SylvainCorlay]

Basically we'll need to have integration checks for things like jupyterlab-pygments to make sure they work in a release.

All there is in jupyterlab-pygments is a list of colors for pygments:

https://github.com/jupyterlab/jupyterlab_pygments/blob/master/jupyterlab_pygments/style.py#L54-L133

What would you environ as a test for that? Should we check that the variables still exist in JupyterLab's CSS?

[blink1073]

What would you environ as a test for that? Should we check that the variables still exist in JupyterLab's CSS?

A notebook that runs and produces DOM nodes that we can inspect properties of.

[SylvainCorlay]

What would you environ as a test for that? Should we check that the variables still exist in JupyterLab's CSS?

A notebook that runs and produces DOM nodes that we can inspect properties of.

This may be a test for nbconvert I guess, but JupyterLab-pygments is merely a pygments style using JupyterLab's CSS variables. (The ones used in the custom codemirror theme).

[blink1073]

I mean running the notebook in the browser

[blink1073]

We need more of that, in general.

[blink1073]

For instance, our release checklist has us manually running this notebook and making sure it works.

[SylvainCorlay]

Ah gotcha. Should I render some code with Pygments (and that style) inline in a notebook?

[blink1073]

Yeah, you can either keep in in the pygments repo and we can target it, or put it here.

[lresende]

If what is being proposed here is to have jupyterlab org for core jupyterlab, and move some of the current extensions to the new org, why not call it jupyterlab-extensions?

If something else is being proposed, could you please clarify...

My personal preference, and mostly for visibility and community building, would be to avoid fragmentation and maybe move things that are not being updated to some sort of attic or archived state. But I don't have strong feelings.

If we need to distinguish between core committers and extension committers, we can still handle that with one org using teams.

[SylvainCorlay]

Yeah, you can either keep in in the pygments repo and we can target it, or put it here.

OK, I will add it first to the jupyterlab_pygments repository. It will be a nice example actually.

[blink1073]

Hi @lresende, thanks for pushing back!

why not call it jupyterlab-extensions?

It might contain server extensions, lsp providers, or something else entirely

we can still handle that with one org using teams

Teams are only really visible internally, and even then I get confused as to who is on what team. Having a separate org makes it much more explicit.

[SylvainCorlay]

Yeah, you can either keep in in the pygments repo and we can target it, or put it here.

I added a test notebook and a screencast in the readme.

Note that jupyterlab_pygments is a dependency of nbconvert 6.0 (which is in pre-release at the moment) so I think it should remain in a Jupyter org.

[fcollonval]

I just created jupyterlab-contrib organization in the hope to initiate a similar dynamic than jupyter_contrib_nbextensions for JupyterLab.

Happy to discuss governance and to add volunteers that want to help in it.

The initial need came from the transfer of maintenance on jupyter-archive.

[fcollonval]

Would it be ok to use the lab logo in that organization picture? It would look like:

[jasongrout]

Any trademark or logo questions should be addressed to the trademark committee: https://jupyter.org/governance/trademarks.html

[fcollonval]

Ant trademark or logo questions should be addressed to the trademark committee: https://jupyter.org/governance/trademarks.html

Thanks Jason for the quick answer. The request has been submitted.

[jtpio]

Thanks @fcollonval for starting this 👍

Wondering whether the organization should maybe mention the word "community" as an alternative to "unofficial"? Something like "JupyterLab Community Extensions & Tools".

[lresende]

Is this new org being part of Jupyter and following Jupyter governance?

[bollwyvl]

Yeah, if one of the requirements is adheres (and links to) the Jupyter Code of Conduct, "community" feels nicer.

And while a moon is, indeed, technically a satellite of a planyt, a nice line art icon of a human-made satellite would have a lot of punch... cubesats have totally changed the education/execution of space missions in the last 10 years, we might want to harken a bit to that.

[lresende]

If this would be an "officially accepted Jupyter org" where all projects are official Jupyter projects, then I would not oppose it, but if we really think about it, the proliferation of multiple org will make what in the past was a cohesive community, to become fragmented into multiple locations which will make sub-projects apart of each other with time.

[fcollonval]

Thanks all for your contributions.

As nicely summarized by Jason in that comment, there are many questions arisen by this; especially regarding maintenance burden, organization management and level of trust user will have.
As rights management on GitHub can only be fine grained control in organization and because the fork of fork of fork nightmares (not even speaking of the multiplication of similar packages on pypi, npm or conda that breaks user experience), I seems the best vessel for ensuring maintenance through various authors - as for example it is done on some official extensions (like @jupyterlab/git).

As I'm not involved in the Jupyter governance and after looking closely on the jupyter_contrib_nbextensions repository strategy, I decided to mimic it. So for now I use the term unofficial to clearly state it is not officially accepted by the Jupyter.org and it should not be more trusted than any code one may find out on the internet. And therefore I did not enforce the Jupyter Code of Conduct.
That said, I will happier if the community (or else?) decides to make it official. Then I fully agree it should follow the Jupyter Code of Conduct. But then questions summarize by Jason must be addressed regarding maintenance, governance and project acceptance as the community will trust the content found in that organization.

To answer more specifically the comments made:

Wondering whether the organization should maybe mention the word "community" as an alternative to "unofficial"? Something like "JupyterLab Community Extensions & Tools".

I did not use the word community and prefer unofficial to not give a false sense of trust.

Is this new org being part of Jupyter and following Jupyter governance?
adheres (and links to) the Jupyter Code of Conduct

For now no. But I would love to have it accept it (note: I don't know what it is needed)

a nice line art icon of a human-made satellite would have a lot of punch

Great idea thanks for it

the proliferation of multiple org will make what in the past was a cohesive community, to become fragmented into multiple locations

That is a risk indeed; especially as the Jupyter ecosystem is already composed of many organizations.

[jtpio]

I did not use the word community and prefer unofficial to not give a false sense of trust.

Alright. I guess it makes sense and closely mimic the jupyter-contrib-nbextensions strategy.

It might just be a personal preference, but the word community might sound more welcoming?

Also as a reference, other ecosystems use this term even if the projects they host are unofficial. For example playwright-community: https://github.com/playwright-community

[jtpio]

Closing as fixed by #126