UIS: use jupyter server

[oliver-sanders]

Investigate using the Jupyter Server for the UIS.

This may solve a few matters, possibly including:

• UIS: allow running on a different host to the Hub #218
• Cylc UI Server does not accept SSL connections #215

Pull requests welcome!

[dpmatthews]

Background info: https://blog.jupyter.org/99-ways-to-extend-the-jupyter-ecosystem-11e5dab7c54

[oliver-sanders]

Have done a bit more work on this - https://github.com/oliver-sanders/cylc-uiserver/tree/jupyter-server

Conclusions so far:

• The Cylc UIS can be converted to a Jupyter Server extension.
• The server takes care of most of the boiler plate for us and does it in a more Jupyter way.
• Running Lab and the UIS in the same server is pretty darned cool.
• We can make the CylcUIS bimodal, either running it as a standalone server or via the Hub.
• Standalone mode is much nicer for single-user and development purposes.
• Running via the Hub is required for multi-user access.
• We can open up the Cylc endpoints to other users without opening up other Jupyter Server endpoints by adding our own authorisation layer (as planned).
• This will fix UIS: allow running on a different host to the Hub #218 and probably Cylc UI Server does not accept SSL connections #215 (would have to read up on that one).

Here's where I've got to so far (mostly for my own benefit):

• Converted the CylcUIServer to a Jupyter Server extension.
• Added fallback to generate userprofile data for hubless use.
• Ran the server in standalone mode (with browser token), got the UI working.
• Created a Hub application for running the server via the Hub (same approach Jupyter Lab uses), got the UI working for the workflow owner.
• Opened up the Cylc endpoints to other users, UI partially working for other users workflow subscriptions when the hub user != uis user cylc-ui#679

Outstanding issues:

• Default URL seems to be /lab rather than /cylc for some reason.
• Other users must first visit the /lab endpoint to authorise passing their details to the hub authentication otherwise the /cylc endpoint will fail in strange ways.
• UI tree view does not work for other users - workflow subscriptions when the hub user != uis user cylc-ui#679
• Need to restore the server shutdown logic (server cannot currently shutdown completely). Done via another PR to Jupyter Server
• Jupyter Server extensions don't like Python namespace packages so I've had to create a separate top level namespace. - Raised PR with Jupyter Server
• Can currently run the server standalone OR via the hub, changing between the two by commenting out some code (related to the previous point). - Raised PR with Jupyter Server
• Jupyter Server doesn't like running behind the cylc command so I've had to run the extension as a Python module python -m <namespace>. - Raised PR with Jupyter Server
• Mysterious GraphQL traceback which doesn't seem to affect anything. Error de-mystified, an enum config issue in cylc-flow.