Allow users to configure clusters

[jcrist]

This adds a mechanism for specifying user configurable cluster
parameters in a declarative way. The set of configurable cluster
parameters (as well as useful metadata like input type/bounds/valid
values, etc...) are exposed via a rest api. This allows creation of nice
interactive GUIs on the client side without requiring client-side
configuration.

[jcrist]

This still needs tests, but now supports a nice user experience on the client side.

Users can query the gateway endpoint to get a description of the valid user options. These are configured by administrators - in this example the config looks like:

c.DaskGateway.cluster_manager_options = Options(
    Integer("worker_cores", min=1, max=4, default=1, label="Worker Cores"),
    Float("worker_memory", min=0.5, max=8, default=1, label="Worker Memory (GB)"),
    Select("environment", options=["basic", "torch", "tensorflow"], label="Conda Environment")
)

The options object supports getting/setting values programmatically (via key or attribute), and does some simple validation on the client side. If the user is using a web based frontend (e.g. jupyter notebooks) and has ipywidgets installed. They also have access to a nice GUI frontend for configuring the cluster.

When they're satisfied with the configuration, they can pass it to gateway.new_cluster to get a new cluster with the specified configuration.

Alternatively, if they know what parameters are available, they can specify them via keyword e.g.

cluster = gateway.new_cluster(worker_cores=2, environment='basic')

This design is flexible, allowing admins to define configuration as needed, and users to get access to introspect the options without any user-side configuration needed. This design also easily supports other graphical frontends, since the option description is a simple json spec (I think a JupyterLab plugin would be nice to have).

[jcrist]

cc @mrocklin, this may interest you.

[mrocklin]

Cool stuff. I think that something like this would be useful as a general dask.distributed issue to make folks from the other cluster managers aware and get their thoughts on how best to handle things like this.