Python js-plugin support for Jetty

[devinrsmith]

The current web-plugin-packager setup only works for our netty/nginx setup; our deephaven-server PyPi package is jetty based, and needs a different solution.

It should be possible to support js-plugin via Jetty; it's likely a much better end-to-end experience than netty/nginx/web-plugin-packager once the proper pieces have been put into place. At the end of the day, the js assets are static, and JettyBackedGrpcServer should be able to serve them up.

Furthermore, the python packages themselves should be able to package up the js assets as data files in the wheel, and the jetty server should have some mechanism for auto-discovery.

It should be as easy as:

$ pip install deephaven-server deephaven-plugin-matplotlib

There are a few different ways to go about implementing this; ie, the hooks that JettyBackedGrpcServer needs. There could be an explicit JsPlugins registration interface; we could use some sort of ServiceLoader pattern; or extend how the python entry points work with the plugins. (Note: we should try and design it to not be python-specific.)

https://github.com/deephaven/deephaven-js-plugins
https://github.com/deephaven/deephaven-plugin-matplotlib
https://github.com/deephaven/deephaven-plugin-plotly

[mofojed]

@jnumainville @mattrunyon this would be a good one to tackle, specifically packaging JS assets with the python plugin. Would improve the DX, if we can just have both the backend/frontend in one folder and can spin it up/test it out right there. Then we can look at moving plugins all to a monorepo.

[devinrsmith]

deephaven-plugin-plotly-express has currently implemented support for packaging the js distribution inside a wheel.

This gets us one step closer to completing this ticket.

Right now, we can manually configure startup to include the js plugin:

START_OPTS="-Ddeephaven.jsPlugins.express=/tmp/dpe/lib/python3.11/site-packages/js/plotly-express/" \
  ./server/jetty-app/build/install/server-jetty/bin/start

Ideally, we could use python entry_points to get this information from the python plugin itself. There was an earlier attempt at creating this structure (https://github.com/deephaven/deephaven-plugin/blob/main/src/deephaven/plugin/js.py), but it was never re-incorporated into deephaven-core (#2908).

As a strawman, deephaven-plugin-plotly-express would have additional logic in deephaven.plot.express:ChartRegistration to tell the DH system about the js content:

class MyJsContent(...):
    @property
    def js_path():
        ...

class ChartRegistration(Registration):
    @classmethod
    def register_into(cls, callback: Callback) -> None:
        callback.register(DeephavenFigureType)
        callback.register(MyJsContent)

, or would have a new entry_point interface:

[options.entry_points]
deephaven.plugin =
    registration_cls = deephaven.plot.express:ChartRegistration
deephaven.js_plugin =
    js_cls = deephaven.plot.express:MyJsContent

that the server could appropriately discover on startup (I believe the former is the better option).

[niloc132]

This is probably closed by #4925?

[mofojed]

Closed with #4925