From 2f48a53eb6600e48e54d8cd33897cc431b21ef93 Mon Sep 17 00:00:00 2001 From: lusop Date: Fri, 25 Mar 2016 14:13:21 -0700 Subject: [PATCH] Add an example on pool usage in the documentation --- docs/concepts.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/concepts.rst b/docs/concepts.rst index 6863cdb4f11fd..c50614143375d 100644 --- a/docs/concepts.rst +++ b/docs/concepts.rst @@ -60,6 +60,16 @@ it a number of worker slots. Tasks can then be associated with one of the existing pools by using the ``pool`` parameter when creating tasks (i.e., instantiating operators). +.. code:: python + + aggregate_db_message_job = BashOperator( + task_id='aggregate_db_message_job', + execution_timeout=timedelta(hours=3), + pool='ep_data_pipeline_db_msg_agg', + bash_command=aggregate_db_message_job_cmd, + dag=dag) + aggregate_db_message_job.set_upstream(wait_for_empty_queue) + The ``pool`` parameter can be used in conjunction with ``priority_weight`` to define priorities in the queue, and which tasks get executed first as slots open up in the