Skip to content

Commit

Permalink
Review how-to guides, notably the patterns guide.
Browse files Browse the repository at this point in the history
Fix #1209.
  • Loading branch information
aaugustin committed Feb 15, 2025
1 parent e934680 commit 667e418
Show file tree
Hide file tree
Showing 8 changed files with 113 additions and 87 deletions.
1 change: 1 addition & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@

intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
"sesame": ("https://django-sesame.readthedocs.io/en/stable/", None),
"werkzeug": ("https://werkzeug.palletsprojects.com/en/stable/", None),
}

Expand Down
17 changes: 9 additions & 8 deletions docs/howto/autoreload.rst
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
Reload on code changes
======================

When developing a websockets server, you may run it locally to test changes.
Unfortunately, whenever you want to try a new version of the code, you must
stop the server and restart it, which slows down your development process.
When developing a websockets server, you are likely to run it locally to test
changes. Unfortunately, whenever you want to try a new version of the code, you
must stop the server and restart it, which slows down your development process.

Web frameworks such as Django or Flask provide a development server that
reloads the application automatically when you make code changes. There is no
such functionality in websockets because it's designed only for production.
Web frameworks such as Django or Flask provide a development server that reloads
the application automatically when you make code changes. There is no equivalent
functionality in websockets because it's designed only for production.

However, you can achieve the same result easily.
However, you can achieve the same result easily with a third-party library and a
shell command.

Install watchdog_ with the ``watchmedo`` shell utility:

Expand All @@ -27,4 +28,4 @@ Run your server with ``watchmedo auto-restart``:
python app.py
This example assumes that the server is defined in a script called ``app.py``
and exits cleanly when receiving the ``SIGTERM`` signal. Adapt it as necessary.
and exits cleanly when receiving the ``SIGTERM`` signal. Adapt as necessary.
11 changes: 6 additions & 5 deletions docs/howto/debugging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ Enable debug logs

websockets logs events with the :mod:`logging` module from the standard library.

It writes to the ``"websockets.server"`` and ``"websockets.client"`` loggers.
It emits logs in the ``"websockets.server"`` and ``"websockets.client"``
loggers.

Enable logs at the ``DEBUG`` level to see exactly what websockets is doing.
You can enable logs at the ``DEBUG`` level to see exactly what websockets does.

If logging isn't configured in your application::

Expand All @@ -24,10 +25,10 @@ If logging is already configured::
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler())

Refer to the :doc:`logging guide <../topics/logging>` for more details on
Refer to the :doc:`logging guide <../topics/logging>` for more information about
logging in websockets.

In addition, you may enable asyncio's `debug mode`_ to see what asyncio is
doing.
You may also enable asyncio's `debug mode`_ to get warnings about classic
pitfalls.

.. _debug mode: https://docs.python.org/3/library/asyncio-dev.html#asyncio-debug-mode
57 changes: 29 additions & 28 deletions docs/howto/django.rst
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ call its APIs in the websockets server.
Now here's how to implement authentication.

.. literalinclude:: ../../example/django/authentication.py
:caption: authentication.py

Let's unpack this code.

Expand All @@ -113,23 +114,25 @@ your settings module.

The connection handler reads the first message received from the client, which
is expected to contain a django-sesame token. Then it authenticates the user
with ``get_user()``, the API for `authentication outside a view`_. If
authentication fails, it closes the connection and exits.
with :func:`~sesame.utils.get_user`, the API provided by django-sesame for
`authentication outside a view`_.

.. _authentication outside a view: https://django-sesame.readthedocs.io/en/stable/howto.html#outside-a-view

When we call an API that makes a database query such as ``get_user()``, we
wrap the call in :func:`~asyncio.to_thread`. Indeed, the Django ORM doesn't
support asynchronous I/O. It would block the event loop if it didn't run in a
separate thread.
If authentication fails, it closes the connection and exits.

When we call an API that makes a database query such as
:func:`~sesame.utils.get_user`, we wrap the call in :func:`~asyncio.to_thread`.
Indeed, the Django ORM doesn't support asynchronous I/O. It would block the
event loop if it didn't run in a separate thread.

Finally, we start a server with :func:`~websockets.asyncio.server.serve`.

We're ready to test!

Save this code to a file called ``authentication.py``, make sure the
``DJANGO_SETTINGS_MODULE`` environment variable is set properly, and start the
websockets server:
Download :download:`authentication.py <../../example/django/authentication.py>`,
make sure the ``DJANGO_SETTINGS_MODULE`` environment variable is set properly,
and start the websockets server:

.. code-block:: console
Expand Down Expand Up @@ -169,7 +172,7 @@ following code in the JavaScript console of the browser:
websocket.onmessage = (event) => console.log(event.data);
If you don't want to import your entire Django project into the websockets
server, you can build a separate Django project with ``django.contrib.auth``,
server, you can create a simpler Django project with ``django.contrib.auth``,
``django-sesame``, a suitable ``User`` model, and a subset of the settings of
the main project.

Expand All @@ -184,11 +187,11 @@ action was made. This may be used for showing notifications to other users.

Many use cases for WebSocket with Django follow a similar pattern.

Set up event bus
................
Set up event stream
...................

We need a event bus to enable communications between Django and websockets.
Both sides connect permanently to the bus. Then Django writes events and
We need an event stream to enable communications between Django and websockets.
Both sides connect permanently to the stream. Then Django writes events and
websockets reads them. For the sake of simplicity, we'll rely on `Redis
Pub/Sub`_.

Expand Down Expand Up @@ -219,14 +222,15 @@ change ``get_redis_connection("default")`` in the code below to the same name.
Publish events
..............

Now let's write events to the bus.
Now let's write events to the stream.

Add the following code to a module that is imported when your Django project
starts. Typically, you would put it in a ``signals.py`` module, which you
would import in the ``AppConfig.ready()`` method of one of your apps:
starts. Typically, you would put it in a :download:`signals.py
<../../example/django/signals.py>` module, which you would import in the
``AppConfig.ready()`` method of one of your apps:

.. literalinclude:: ../../example/django/signals.py

:caption: signals.py
This code runs every time the admin saves a ``LogEntry`` object to keep track
of a change. It extracts interesting data, serializes it to JSON, and writes
an event to Redis.
Expand Down Expand Up @@ -256,13 +260,13 @@ We need to add several features:

* Keep track of connected clients so we can broadcast messages.
* Tell which content types the user has permission to view or to change.
* Connect to the message bus and read events.
* Connect to the message stream and read events.
* Broadcast these events to users who have corresponding permissions.

Here's a complete implementation.

.. literalinclude:: ../../example/django/notifications.py

:caption: notifications.py
Since the ``get_content_types()`` function makes a database query, it is
wrapped inside :func:`asyncio.to_thread()`. It runs once when each WebSocket
connection is open; then its result is cached for the lifetime of the
Expand All @@ -273,13 +277,10 @@ The connection handler merely registers the connection in a global variable,
associated to the list of content types for which events should be sent to
that connection, and waits until the client disconnects.

The ``process_events()`` function reads events from Redis and broadcasts them
to all connections that should receive them. We don't care much if a sending a
notification fails — this happens when a connection drops between the moment
we iterate on connections and the moment the corresponding message is sent —
so we start a task with for each message and forget about it. Also, this means
we're immediately ready to process the next event, even if it takes time to
send a message to a slow client.
The ``process_events()`` function reads events from Redis and broadcasts them to
all connections that should receive them. We don't care much if a sending a
notification fails. This happens when a connection drops between the moment we
iterate on connections and the moment the corresponding message is sent.

Since Redis can publish a message to multiple subscribers, multiple instances
of this server can safely run in parallel.
Expand All @@ -290,4 +291,4 @@ Does it scale?
In theory, given enough servers, this design can scale to a hundred million
clients, since Redis can handle ten thousand servers and each server can
handle ten thousand clients. In practice, you would need a more scalable
message bus before reaching that scale, due to the volume of messages.
message stream before reaching that scale, due to the volume of messages.
39 changes: 24 additions & 15 deletions docs/howto/extensions.rst
Original file line number Diff line number Diff line change
@@ -1,30 +1,39 @@
Write an extension
==================

.. currentmodule:: websockets.extensions
.. currentmodule:: websockets

During the opening handshake, WebSocket clients and servers negotiate which
extensions_ will be used with which parameters. Then each frame is processed
by extensions before being sent or after being received.
extensions_ will be used and with which parameters.

.. _extensions: https://datatracker.ietf.org/doc/html/rfc6455.html#section-9

As a consequence, writing an extension requires implementing several classes:
Then, each frame is processed before being sent and after being received
according to the extensions that were negotiated.

* Extension Factory: it negotiates parameters and instantiates the extension.
Writing an extension requires implementing at least two classes, an extension
factory and an extension. They inherit from base classes provided by websockets.

Clients and servers require separate extension factories with distinct APIs.
Extension factory
-----------------

Extension factories are the public API of an extension.
An extension factory negotiates parameters and instantiates the extension.

* Extension: it decodes incoming frames and encodes outgoing frames.
Clients and servers require separate extension factories with distinct APIs.
Base classes are :class:`~extensions.ClientExtensionFactory` and
:class:`~extensions.ServerExtensionFactory`.

If the extension is symmetrical, clients and servers can use the same
class.
Extension factories are the public API of an extension. Extensions are enabled
with the ``extensions`` parameter of :func:`~asyncio.client.connect` or
:func:`~asyncio.server.serve`.

Extensions are initialized by extension factories, so they don't need to be
part of the public API of an extension.
Extension
---------

websockets provides base classes for extension factories and extensions.
See :class:`ClientExtensionFactory`, :class:`ServerExtensionFactory`,
and :class:`Extension` for details.
An extension decodes incoming frames and encodes outgoing frames.

If the extension is symmetrical, clients and servers can use the same class. The
base class is :class:`~extensions.Extension`.

Since extensions are initialized by extension factories, they don't need to be
part of the public API of an extension.
9 changes: 3 additions & 6 deletions docs/howto/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,19 @@ Configure websockets securely in production.

encryption

If you're stuck, perhaps you'll find the answer here.
These guides will help you design and build your application.

.. toctree::
:maxdepth: 2

patterns

This guide will help you integrate websockets into a broader system.

.. toctree::

django

Upgrading from the legacy :mod:`asyncio` implementation to the new one?
Read this.

.. toctree::
:maxdepth: 2

upgrade

Expand Down
64 changes: 39 additions & 25 deletions docs/howto/patterns.rst
Original file line number Diff line number Diff line change
@@ -1,46 +1,52 @@
Patterns
========
Design a WebSocket application
==============================

.. currentmodule:: websockets

Here are typical patterns for processing messages in a WebSocket server or
client. You will certainly implement some of them in your application.
WebSocket server or client applications follow common patterns. This guide
describes patterns that you're likely to implement in your application.

This page gives examples of connection handlers for a server. However, they're
also applicable to a client, simply by assuming that ``websocket`` is a
connection created with :func:`~asyncio.client.connect`.
All examples are connection handlers for a server. However, they would also
apply to a client, assuming that ``websocket`` is a connection created with
:func:`~asyncio.client.connect`.

WebSocket connections are long-lived. You will usually write a loop to process
several messages during the lifetime of a connection.
.. admonition:: WebSocket connections are long-lived.
:class: tip

Consumer
--------
You need a loop to process several messages during the lifetime of a
connection.

Consumer pattern
----------------

To receive messages from the WebSocket connection::

async def consumer_handler(websocket):
async for message in websocket:
await consumer(message)
await consume(message)

In this example, ``consumer()`` is a coroutine implementing your business
logic for processing a message received on the WebSocket connection. Each
message may be :class:`str` or :class:`bytes`.
In this example, ``consume()`` is a coroutine implementing your business logic
for processing a message received on the WebSocket connection.

Iteration terminates when the client disconnects.

Producer
--------
Producer pattern
----------------

To send messages to the WebSocket connection::

from websockets.exceptions import ConnectionClosed

async def producer_handler(websocket):
while True:
message = await producer()
await websocket.send(message)
try:
while True:
message = await produce()
await websocket.send(message)
except ConnectionClosed:
break

In this example, ``producer()`` is a coroutine implementing your business
logic for generating the next message to send on the WebSocket connection.
Each message must be :class:`str` or :class:`bytes`.
In this example, ``produce()`` is a coroutine implementing your business logic
for generating the next message to send on the WebSocket connection.

Iteration terminates when the client disconnects because
:meth:`~asyncio.server.ServerConnection.send` raises a
Expand All @@ -51,8 +57,12 @@ Consumer and producer
---------------------

You can receive and send messages on the same WebSocket connection by
combining the consumer and producer patterns. This requires running two tasks
in parallel::
combining the consumer and producer patterns.

This requires running two tasks in parallel. The simplest option offered by
:mod:`asyncio` is::

import asyncio

async def handler(websocket):
await asyncio.gather(
Expand Down Expand Up @@ -99,6 +109,10 @@ connect and unregister them when they disconnect::
This example maintains the set of connected clients in memory. This works as
long as you run a single process. It doesn't scale to multiple processes.

If you just need the set of connected clients, as in this example, use the
:attr:`~asyncio.server.Server.connections` property of the server. This pattern
is needed only when recording additional information about each client.

Publish–subscribe
-----------------

Expand Down
2 changes: 2 additions & 0 deletions docs/project/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ New features
Improvements
............

* Refreshed several how-to guides and topic guides.

* Added type overloads for the ``decode`` argument of
:meth:`~asyncio.connection.Connection.recv`. This may simplify static typing.

Expand Down

0 comments on commit 667e418

Please sign in to comment.