Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add flow control section to publish overview #129

Merged
merged 5 commits into from
Jul 10, 2020
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/publisher/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,44 @@ You can also attach a callback to the future:
future.add_done_callback(callback)


Publish Flow Control
--------------------

If publishing large amounts of messages or very large messages in quick
succession, some of the publish requests might time out, especially if the
bandwidth available is limited. To mitigate this the client can be
configured with custom :class:`~.pubsub_v1.types.PublishFlowControl` settings.

You can configure the maximum desired number of messages and their maximum total
plamut marked this conversation as resolved.
Show resolved Hide resolved
size, as well as the action that should be taken when the threshold is reached.

.. code-block:: python

from google.cloud import pubsub_v1

client = pubsub_v1.PublisherClient(
publisher_options=pubsub_v1.types.PublisherOptions(
flow_control=pubsub_v1.types.PublishFlowControl(
message_limit=500,
byte_limit=2 * 1024 * 1024,
limit_exceeded_behavior=pubsub_v1.types.LimitExceededBehavior.BLOCK,
),
),
)

The action to be taken on overflow can be one of the following:

* :attr:`~.pubsub_v1.types.LimitExceededBehavior.IGNORE` (default): Ignore the
overflow and continue publishing the messages as normal.
* :attr:`~.pubsub_v1.types.LimitExceededBehavior.ERROR`: Raise
:exc:`~.pubsub_v1.publisher.exceptions.FlowControlLimitError` and reject the message.
* :attr:`~.pubsub_v1.types.LimitExceededBehavior.BLOCK`: Temporarily block in the
:meth:`~.pubsub_v1.publisher.client.Client.publish` method until there is
enough capacity available. Excessive messages are queued internally before
plamut marked this conversation as resolved.
Show resolved Hide resolved
their corresponding publish requests are actually made, reducing the chance of
network timeouts.


API Reference
-------------

Expand Down