Brainstorming: new use of entry_points for composability of Jupyter ecosystem software?

[consideRatio]

Code in the Jupyter ecosystem sometimes make use of Python entry_points to identity other code to integrate with. I

Examples of entry_points

• jupyter-server-proxy processes registering themselves to later be identified by jupyter-server-proxy.
  • Via the entrypoint jupyter_serverproxy_servers
• JupyterHub Spawners registering themselves
  • jupyterhub.spawners, jupyterhub.proxies, and jupyterhub.authenticators are registrable and recognized by JupyterHub.
• The Littlest JupyterHub plugins registers themselves via the tljh entry point.

New uses of entry_points?

• Maybe jupyterhub could regonize JupyterHub managed services via entry_points so that one didn't even declare c.JupyterHub.services but just installed the managed service Python package such as https://github.com/jupyterhub/jupyterhub-idle-culler?

• Maybe jupyterhub-singleuser or jupyter_server could use entry_points to discover other code to help address challenges below in a way that was helpful.

  • Doing things as root before starting jupyterhub-singleuser as a non-root process: https://github.com/yuvipanda/jupyterhub-roothooks
  • Capturing logs by user server processes from an outer process: Enable users to access the jupyter server logs jupyter-server/jupyter_server#684

Overall I think there can be complexity with these ideas making them not worth going for, but even if these ideas doesn't turn out to lead somewhere I figure maybe we could benefit from thinking about this a bit together.

[minrk]

In general, I don't love using a discovery mechanism (entrypoints) to also trigger code execution, especially in an environment like JupyterHub, which often runs as root. Just like with most plugin systems (spawners, etc.) making something available still requires an explicit step to use it.

But it does potentially make sense as an avenue to simplify loading a plugin such as the idle-culler to shift more code over there, so a config file might just say services.append("jupyterhub-idle-culler") and a hook in jupyterhub_idle_culler takes care of the rest. Do you think that is enough? What do you think that example would look like compared to what's required now? How would configuration of the service be handled?

Maybe jupyterhub-singleuser or jupyter_server could use entry_points

jupyter-server already has an extension mechanism that works very well and handles discovery, loading, etc. I don't think we need anything new to add there, and don't think we should have a jupyterhub-specific extension mechanism for something that already exists. In fact, I'm trying to get jupyterhub-singleuser down to just a server extension in jupyterhub/jupyterhub#3888.

I think we could possibly implement both the root hooks and the log capture as standard server extensions (using os.dup2).

[minrk]

I forgot to mention - I'm absolutely 👍 on entrypoints as a discovery mechanism anywhere we find it's helpful, so anywhere we do find a discovery problem I think it's a good thing to start with.