Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Dask How to guide #4234

Merged
merged 10 commits into from
Dec 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added doc/_static/images/asyncify.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/_static/images/dask-dashboard-empty.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/_static/images/dask_fibonacci_queue.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions doc/_static/logos/dask-logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
79 changes: 44 additions & 35 deletions doc/how_to/callbacks/async.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,22 @@
# Use Asynchronous Callbacks

This guide addresses how to leverage asynchronous callbacks to run I/O bound tasks in parallel.
This guide addresses how to leverage asynchronous callbacks to run I/O bound tasks in parallel. This technique is also beneficial for CPU bound tasks that release the GIL.

```{admonition} Prerequisites
1. Python has natively supported asynchronous functions since version 3.5, for a quick overview of some of the concepts involved see [the Python documentation](https://docs.python.org/3/library/asyncio-task.html).
You can use `async` function with event handlers like `on_click` as well as the reactive apis `.bind`, `.depends` and `.watch`.

You can also schedule asynchronous periodic callbacks with `pn.state.add_periodic_callback` as well as run `async` functions directly with `pn.state.execute`.

```{admonition} Asyncio
For a quick overview of the most important `asyncio` concepts see [the Python documentation](https://docs.python.org/3/library/asyncio-task.html).
```

```{admonition} Bokeh Models
It is important to note that asynchronous callbacks operate without locking the underlying Bokeh Document, which means Bokeh models cannot be safely modified by default. Usually this is not an issue because modifying Panel components appropriately schedules updates to underlying Bokeh models, however in cases where we want to modify a Bokeh model directly, e.g. when embedding and updating a Bokeh plot in a Panel application we explicitly have to decorate the asynchronous callback with `pn.io.with_lock` (see example below).
```

---

## `.param.watch`
## `on_click`

One of the major benefits of leveraging async functions is that it is simple to write callbacks which will perform some longer running IO tasks in the background. Below we simulate this by creating a `Button` which will update some text when it starts and finishes running a long-running background task (here simulated using `asyncio.sleep`. If you are running this in the notebook you will note that you can start multiple tasks and it will update the text immediately but continue in the background:

Expand All @@ -30,36 +39,7 @@ button.on_click(run_async)
pn.Row(button, text)
```

Note that `on_click` is simple one way of registering an asynchronous callback, but the more flexible `.param.watch` is also supported. Scheduling asynchronous periodic callbacks can be done with `pn.state.add_periodic_callback`.

It is important to note that asynchronous callbacks operate without locking the underlying Bokeh Document, which means Bokeh models cannot be safely modified by default. Usually this is not an issue because modifying Panel components appropriately schedules updates to underlying Bokeh models, however in cases where we want to modify a Bokeh model directly, e.g. when embedding and updating a Bokeh plot in a Panel application we explicitly have to decorate the asynchronous callback with `pn.io.with_lock`.

```{pyodide}
import numpy as np
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource

button = pn.widgets.Button(name='Click me!')

p = figure(width=500, height=300)
cds = ColumnDataSource(data={'x': [0], 'y': [0]})
p.line(x='x', y='y', source=cds)
pane = pn.pane.Bokeh(p)

@pn.io.with_lock
async def stream(event):
await asyncio.sleep(1)
x, y = cds.data['x'][-1], cds.data['y'][-1]
cds.stream({'x': list(range(x+1, x+6)), 'y': y+np.random.randn(5).cumsum()})
pane.param.trigger('object')

# Equivalent to `.on_click` but shown
button.param.watch(stream, 'clicks')

pn.Row(button, pane)
```

## `pn.bind`
## `.bind`

```{pyodide}
widget = pn.widgets.IntSlider(start=0, end=10)
Expand All @@ -80,7 +60,9 @@ pn.Column(widget, pn.bind(get_img, widget))

In this example Panel will invoke the function and update the output when the function returns while leaving the process unblocked for the duration of the `aiohttp` request.

The equivalent can be written using `.param.watch` as:
## `.watch`

The app from the section above can be written using `.param.watch` as:

```{pyodide}
widget = pn.widgets.IntSlider(start=0, end=10)
Expand All @@ -107,6 +89,33 @@ pn.Column(widget, image)

In this example Param will await the asynchronous function and the image will be updated when the request completes.

## Bokeh models with `pn.io.with_lock`

```{pyodide}
import numpy as np
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource

button = pn.widgets.Button(name='Click me!')

p = figure(width=500, height=300)
cds = ColumnDataSource(data={'x': [0], 'y': [0]})
p.line(x='x', y='y', source=cds)
pane = pn.pane.Bokeh(p)

@pn.io.with_lock
async def stream(event):
await asyncio.sleep(1)
x, y = cds.data['x'][-1], cds.data['y'][-1]
cds.stream({'x': list(range(x+1, x+6)), 'y': y+np.random.randn(5).cumsum()})
pane.param.trigger('object')

# Equivalent to `.on_click` but shown
button.param.watch(stream, 'clicks')

pn.Row(button, pane)
```

## Related Resources

- See the related [How-to > Link Parameters with Callbacks API](../links/index.md) guides, including [How to > Create Low-Level Python Links Using `.watch`](../links/watchers.md).
51 changes: 0 additions & 51 deletions doc/how_to/concurrency/async.md

This file was deleted.

Loading
Loading