Skip to content

Commit

Permalink
Add Dask How to guide (#4234)
Browse files Browse the repository at this point in the history
  • Loading branch information
MarcSkovMadsen authored Dec 27, 2023
1 parent 87cef45 commit a1e32c0
Show file tree
Hide file tree
Showing 9 changed files with 366 additions and 89 deletions.
Binary file added doc/_static/images/asyncify.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/_static/images/dask-dashboard-empty.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/_static/images/dask_fibonacci_queue.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions doc/_static/logos/dask-logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
79 changes: 44 additions & 35 deletions doc/how_to/callbacks/async.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,22 @@
# Use Asynchronous Callbacks

This guide addresses how to leverage asynchronous callbacks to run I/O bound tasks in parallel.
This guide addresses how to leverage asynchronous callbacks to run I/O bound tasks in parallel. This technique is also beneficial for CPU bound tasks that release the GIL.

```{admonition} Prerequisites
1. Python has natively supported asynchronous functions since version 3.5, for a quick overview of some of the concepts involved see [the Python documentation](https://docs.python.org/3/library/asyncio-task.html).
You can use `async` function with event handlers like `on_click` as well as the reactive apis `.bind`, `.depends` and `.watch`.

You can also schedule asynchronous periodic callbacks with `pn.state.add_periodic_callback` as well as run `async` functions directly with `pn.state.execute`.

```{admonition} Asyncio
For a quick overview of the most important `asyncio` concepts see [the Python documentation](https://docs.python.org/3/library/asyncio-task.html).
```

```{admonition} Bokeh Models
It is important to note that asynchronous callbacks operate without locking the underlying Bokeh Document, which means Bokeh models cannot be safely modified by default. Usually this is not an issue because modifying Panel components appropriately schedules updates to underlying Bokeh models, however in cases where we want to modify a Bokeh model directly, e.g. when embedding and updating a Bokeh plot in a Panel application we explicitly have to decorate the asynchronous callback with `pn.io.with_lock` (see example below).
```

---

## `.param.watch`
## `on_click`

One of the major benefits of leveraging async functions is that it is simple to write callbacks which will perform some longer running IO tasks in the background. Below we simulate this by creating a `Button` which will update some text when it starts and finishes running a long-running background task (here simulated using `asyncio.sleep`. If you are running this in the notebook you will note that you can start multiple tasks and it will update the text immediately but continue in the background:

Expand All @@ -30,36 +39,7 @@ button.on_click(run_async)
pn.Row(button, text)
```

Note that `on_click` is simple one way of registering an asynchronous callback, but the more flexible `.param.watch` is also supported. Scheduling asynchronous periodic callbacks can be done with `pn.state.add_periodic_callback`.

It is important to note that asynchronous callbacks operate without locking the underlying Bokeh Document, which means Bokeh models cannot be safely modified by default. Usually this is not an issue because modifying Panel components appropriately schedules updates to underlying Bokeh models, however in cases where we want to modify a Bokeh model directly, e.g. when embedding and updating a Bokeh plot in a Panel application we explicitly have to decorate the asynchronous callback with `pn.io.with_lock`.

```{pyodide}
import numpy as np
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource
button = pn.widgets.Button(name='Click me!')
p = figure(width=500, height=300)
cds = ColumnDataSource(data={'x': [0], 'y': [0]})
p.line(x='x', y='y', source=cds)
pane = pn.pane.Bokeh(p)
@pn.io.with_lock
async def stream(event):
await asyncio.sleep(1)
x, y = cds.data['x'][-1], cds.data['y'][-1]
cds.stream({'x': list(range(x+1, x+6)), 'y': y+np.random.randn(5).cumsum()})
pane.param.trigger('object')
# Equivalent to `.on_click` but shown
button.param.watch(stream, 'clicks')
pn.Row(button, pane)
```

## `pn.bind`
## `.bind`

```{pyodide}
widget = pn.widgets.IntSlider(start=0, end=10)
Expand All @@ -80,7 +60,9 @@ pn.Column(widget, pn.bind(get_img, widget))

In this example Panel will invoke the function and update the output when the function returns while leaving the process unblocked for the duration of the `aiohttp` request.

The equivalent can be written using `.param.watch` as:
## `.watch`

The app from the section above can be written using `.param.watch` as:

```{pyodide}
widget = pn.widgets.IntSlider(start=0, end=10)
Expand All @@ -107,6 +89,33 @@ pn.Column(widget, image)

In this example Param will await the asynchronous function and the image will be updated when the request completes.

## Bokeh models with `pn.io.with_lock`

```{pyodide}
import numpy as np
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource
button = pn.widgets.Button(name='Click me!')
p = figure(width=500, height=300)
cds = ColumnDataSource(data={'x': [0], 'y': [0]})
p.line(x='x', y='y', source=cds)
pane = pn.pane.Bokeh(p)
@pn.io.with_lock
async def stream(event):
await asyncio.sleep(1)
x, y = cds.data['x'][-1], cds.data['y'][-1]
cds.stream({'x': list(range(x+1, x+6)), 'y': y+np.random.randn(5).cumsum()})
pane.param.trigger('object')
# Equivalent to `.on_click` but shown
button.param.watch(stream, 'clicks')
pn.Row(button, pane)
```

## Related Resources

- See the related [How-to > Link Parameters with Callbacks API](../links/index.md) guides, including [How to > Create Low-Level Python Links Using `.watch`](../links/watchers.md).
51 changes: 0 additions & 51 deletions doc/how_to/concurrency/async.md

This file was deleted.

Loading

0 comments on commit a1e32c0

Please sign in to comment.