Many network errors resulting from dashboard URL persisting in jupyterlab workspace

[scottyhq]

What happened:
labextension dashboard launcher url saved in jupyterlab workspace, causing repeating 503, 403 & 404 network errors every second after a cluster is shut down or the next time users open up a jupyterhub session:

Errors such as this appear in the console every few seconds:

dask-logo.svg:1 GET https://staging.aws-uswest2.pangeo.io/services/dask-gateway/clusters/icesat2-staging.24e564509dbd475abd71272306fbba06/statics/images/dask-logo.svg?1602869144274 503
Image (async)


vendors~main.0696ed76febe5543ce98.js:2 GET https://aws-uswest2.pangeo.io/user/scottyhq/dask/dashboard/5b22bb4e-80fa-4803-84cd-5dc7fdd97f06/individual-plots.json?1602875598184 404 (or 403)

As far as I can tell these errors are coming from the dashboard URL persisting in /home/jovyan/.jupyter/lab/workspaces/*:

{"data":{"layout-restorer:data":{"main":{"dock":{"type":"tab-area","currentIndex":0,"widgets":["terminal:1"]},"mode":"multiple-document","current":"terminal:1"},"left":{"collapsed":false,"current":"filebrowser","widgets":["filebrowser","running-sessions","dask-dashboard-launcher","command-palette","jp-property-inspector","tab-manager","extensionmanager.main-view"]},"right":{"collapsed":true,"widgets":[]}},"file-browser-filebrowser:cwd":{"path":""},"notebook:hvplot-bouncing-axes.ipynb":{"data":{"path":"hvplot-bouncing-axes.ipynb","factory":"Notebook"}},"dask-dashboard-launcher":{"url":"https://staging.aws-uswest2.pangeo.io/services/dask-gateway/clusters/icesat2-staging.24e564509dbd475abd71272306fbba06/status","cluster":""},"terminal:1":{"data":{"name":"1"}}},"metadata":{"id":"/lab"}}

key entry seeming to be this if the status url is manually copied into the sidebar "dask-dashboard-launcher":{"url":"https://staging.aws-uswest2.pangeo.io/services/dask-gateway/clusters/icesat2-staging.24e564509dbd475abd71272306fbba06/status","cluster":""} Or, if a Gateway cluster is created from the lab-extension this config remains: "dask-dashboard-launcher":{"url":"dask/dashboard/5b22bb4e-80fa-4803-84cd-5dc7fdd97f06","cluster":""}

What you expected to happen:
I expect the URL to disappear from workspace config when a kernel is restarted, when I manually delete the URL from the dask-labextension dashboard, or when I logout.

Environment: ( jupyterhub on kubernetes)

python                    3.7.8           h6f2ec95_1_cpython    conda-forge
dask-core                 2.30.0                     py_0    conda-forge
dask-gateway              0.8.0            py37hc8dfbb8_0    conda-forge
dask-glm                  0.2.0                      py_1    conda-forge
dask-kubernetes           0.11.0                     py_0    conda-forge
dask-labextension         3.0.0                      py_0    conda-forge

jupyter-server-proxy      1.5.0                      py_0    conda-forge
jupyterhub-singleuser     1.1.0            py37hc8dfbb8_4    conda-forge
jupyterlab                2.2.8                      py_0    conda-forge
jupyterlab_server         1.2.0                      py_0    conda-forge

Operating System:
Install method (conda, pip, source): conda-forge

Current workaround:
Go to the https://aws-uswest2.pangeo.io/user/scottyhq/lab?reset in order to remove the last URL from the workspace. Would be nice to have the URL reset whenever a kernel is restarted, user logs out, or cluster is shut down.

Related issues
cc @ian-r-rose , @TomAugspurger , @jcrist
#20, #135, pangeo-data/pangeo-cloud-federation#786

[jsignell]

My understanding is that the dashboard url needs to be saved to the workspace so that if you refresh the browser tab, the same panes are open and the same client is connected. So like you say, we'd need to explicitly unsave the url from the state.

We could try to do this in the cases that you list, or we could give up on a client after it fails a certain number of times. Do you think that would be a reasonable solution?

[scottyhq]

thanks for the followup @jsignell! That all makes sense. both of the solutions you mention seem like reasonable things to do!