Add --match flag to work queues documentation #7768

riquelmev · 2022-12-05T19:34:53Z

Added documentation about the --match flag in agent configuration to work-queues.md.
Preview here: https://deploy-preview-7768--prefect-orion.netlify.app/concepts/work-queues
Closes #7511

This pull request references any related issue by including "closes <link to issue>"
This pull request includes tests or only affects documentation.
This pull request includes a label categorizing the change e.g. fix, feature, enhancement

netlify · 2022-12-05T19:35:09Z

✅ Deploy Preview for prefect-orion ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`59327e6`
🔍 Latest deploy log	https://app.netlify.com/sites/prefect-orion/deploys/63b881725a345f00083c594a
😎 Deploy Preview	https://deploy-preview-7768--prefect-orion.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

tpdorsey

Looks good. A few minor suggested edits. Thanks for the contribution!

tpdorsey · 2022-12-05T22:12:38Z

docs/concepts/work-queues.md

@@ -49,6 +49,7 @@ Configuration parameters you can specify when starting an agent include:
 | --run-once        | Only run agent polling once. By default, the agent runs forever. |
 | --prefetch-seconds | The amount of time before a flow run's scheduled start time to begin submission. Default is the value of `PREFECT_AGENT_PREFETCH_SECONDS`. |
 | --hide-welcome   | Do not display the startup ASCII art for the agent process.                          |
+| --match | Start all queues that match the given match string


Let's make clear here that this param is about matching prefixes, and will ignore queues passed with -q.
How about:
| -- match QUEUE_PREFIX | Polls for queues matching the given QUEUE_PREFIX. Note: this option overrides anything passed with -q (or --work-queue)

The doc format above with QUEUE_PREFIX is not that consistent, with only the --api TEXT being in this form.. presumably to save space. (-q and --prefetch-seconds could include the param_name). Just a note for future PRs

If it's going to cause -q to be ignored we should throw an error if you try to use both.

That's much better / safer!

Sorry I was wrong, I read downstream code to get that behavior. But it actually already prevents you from passing both, as @madkinsz suggested:

"Only one of work_queues, match, or tags can be provided."

gets printed if the user attempts to pass more than 1. (you can remove the Note: here)

tpdorsey · 2022-12-05T22:25:13Z

docs/concepts/work-queues.md

@@ -94,6 +97,17 @@ In this case, Prefect automatically created a new `my-queue` work queue.

 By default, the agent polls the API specified by the `PREFECT_API_URL` environment variable. To configure the agent to poll from a different server location, use the `--api` flag, specifying the URL of the server.

+In addition, Agents can match multiple work queues by providing a --match string instead of specifying all of the work queues. The agent will poll every work queue with a name that starts with the given string.


Suggested change

In addition, Agents can match multiple work queues by providing a --match string instead of specifying all of the work queues. The agent will poll every work queue with a name that starts with the given string.

In addition, agents can match multiple work queues by providing a `--match` string instead of specifying all of the work queues. The agent will poll every work queue with a name that starts with the given string.

Also add: New queues matching this prefix will be found by the agent without needing to restart it.

zanieb · 2023-01-06T19:49:26Z

@tpdorsey can you review this again for release?

docs/concepts/work-queues.md

Co-authored-by: Terrence Dorsey <terrend@mishu.com>

Co-authored-by: vicente <vicente@getbluesky.io> Co-authored-by: Michael Adkins <contact@zanie.dev> Co-authored-by: Terrence Dorsey <terrend@mishu.com>

…r-builds-release * 'main' of https://github.com/ddelange/prefect: (131 commits) Fix docker-builds.yaml templating syntax (PrefectHQ#8114) Rename Worker pools -> work pools (PrefectHQ#8107) Build multi-arch images for commits on main (PrefectHQ#7900) [Issue PrefectHQ#456] Change example of `infra_override` in docs/concepts/deployment (PrefectHQ#8101) Bump @playwright/test from 1.29.1 to 1.29.2 in /orion-ui (PrefectHQ#8105) Bump @prefecthq/orion-design from 1.1.53 to 1.1.54 in /orion-ui (PrefectHQ#8104) Update deployment docs to include tag and idempotency key (PrefectHQ#7771) Add `BaseWorker` and `ProcessWorker` (PrefectHQ#7996) Add Peyton and Serina as global code owners (PrefectHQ#8098) Add release notes for 2.7.7 (PrefectHQ#8091) Add youtube badge (PrefectHQ#8089) Adds `MAX_RRULE_LENGTH` (PrefectHQ#7762) Limit task run cache key size (PrefectHQ#7275) Add --match flag to work queues documentation (PrefectHQ#7768) Modify disable ssl setting tests to allow any for headers and timeout (PrefectHQ#8086) Add test for allow_failure and quote (PrefectHQ#8055) Adds `experimental_field` decorator (PrefectHQ#8066) add docs on migrating block documents (PrefectHQ#8085) Add Redoc documentation for REST API reference (PrefectHQ#7503) Allow disabling SSL verification (PrefectHQ#7850) ...

…refect into docker-builds-consolidation * 'docker-builds-release' of https://github.com/ddelange/prefect: (131 commits) Fix docker-builds.yaml templating syntax (PrefectHQ#8114) Rename Worker pools -> work pools (PrefectHQ#8107) Build multi-arch images for commits on main (PrefectHQ#7900) [Issue PrefectHQ#456] Change example of `infra_override` in docs/concepts/deployment (PrefectHQ#8101) Bump @playwright/test from 1.29.1 to 1.29.2 in /orion-ui (PrefectHQ#8105) Bump @prefecthq/orion-design from 1.1.53 to 1.1.54 in /orion-ui (PrefectHQ#8104) Update deployment docs to include tag and idempotency key (PrefectHQ#7771) Add `BaseWorker` and `ProcessWorker` (PrefectHQ#7996) Add Peyton and Serina as global code owners (PrefectHQ#8098) Add release notes for 2.7.7 (PrefectHQ#8091) Add youtube badge (PrefectHQ#8089) Adds `MAX_RRULE_LENGTH` (PrefectHQ#7762) Limit task run cache key size (PrefectHQ#7275) Add --match flag to work queues documentation (PrefectHQ#7768) Modify disable ssl setting tests to allow any for headers and timeout (PrefectHQ#8086) Add test for allow_failure and quote (PrefectHQ#8055) Adds `experimental_field` decorator (PrefectHQ#8066) add docs on migrating block documents (PrefectHQ#8085) Add Redoc documentation for REST API reference (PrefectHQ#7503) Allow disabling SSL verification (PrefectHQ#7850) ...

Co-authored-by: vicente <vicente@getbluesky.io> Co-authored-by: Michael Adkins <contact@zanie.dev> Co-authored-by: Terrence Dorsey <terrend@mishu.com>

vicente and others added 4 commits December 5, 2022 09:46

added match documentation

e781d0c

Merge branch 'PrefectHQ:main' into work_queues_docs

0485363

added back divs

2e71277

added punctuation

88170a2

riquelmev requested a review from tpdorsey as a code owner December 5, 2022 19:34

tpdorsey reviewed Dec 5, 2022

View reviewed changes

tpdorsey added docs v2 labels Dec 5, 2022

tpdorsey changed the title ~~Adding --match flag documentation to work-queues.md~~ Add --match flag to work queues documentation Dec 8, 2022

vicente added 2 commits December 10, 2022 22:36

addressed comments

47bf9eb

fixed spacing

c183387

riquelmev requested a review from tpdorsey December 13, 2022 09:53

zanieb mentioned this pull request Dec 18, 2022

Add documentation for agent --match argument #7511

Closed

1 task

tpdorsey approved these changes Jan 6, 2023

View reviewed changes

docs/concepts/work-queues.md Outdated Show resolved Hide resolved

docs/concepts/work-queues.md Outdated Show resolved Hide resolved

docs/concepts/work-queues.md Outdated Show resolved Hide resolved

Apply suggestions from code review

59327e6

Co-authored-by: Terrence Dorsey <terrend@mishu.com>

zanieb merged commit d01ca38 into PrefectHQ:main Jan 6, 2023

github-actions bot pushed a commit that referenced this pull request Jan 6, 2023

Add --match flag to work queues documentation (#7768)

4afa448

Co-authored-by: vicente <vicente@getbluesky.io> Co-authored-by: Michael Adkins <contact@zanie.dev> Co-authored-by: Terrence Dorsey <terrend@mishu.com>

github-actions bot pushed a commit that referenced this pull request Jan 6, 2023

Add --match flag to work queues documentation (#7768)

80d1e0f

Co-authored-by: vicente <vicente@getbluesky.io> Co-authored-by: Michael Adkins <contact@zanie.dev> Co-authored-by: Terrence Dorsey <terrend@mishu.com>

masonmenges pushed a commit that referenced this pull request Jan 10, 2023

Add --match flag to work queues documentation (#7768)

3374287

Co-authored-by: vicente <vicente@getbluesky.io> Co-authored-by: Michael Adkins <contact@zanie.dev> Co-authored-by: Terrence Dorsey <terrend@mishu.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --match flag to work queues documentation #7768

Add --match flag to work queues documentation #7768

riquelmev commented Dec 5, 2022 •

edited by zanieb

Loading

netlify bot commented Dec 5, 2022 •

edited

Loading

tpdorsey left a comment

tpdorsey Dec 5, 2022

kevin868 Dec 8, 2022

kevin868 Dec 8, 2022

zanieb Dec 8, 2022

kevin868 Dec 8, 2022

kevin868 Dec 12, 2022

tpdorsey Dec 5, 2022

kevin868 Dec 8, 2022

zanieb commented Jan 6, 2023

	\| --match \| Start all queues that match the given match string
	\| --match \| Start all queues that match the given match string. \|

		@@ -94,6 +97,17 @@ In this case, Prefect automatically created a new `my-queue` work queue.

		By default, the agent polls the API specified by the `PREFECT_API_URL` environment variable. To configure the agent to poll from a different server location, use the `--api` flag, specifying the URL of the server.

		In addition, Agents can match multiple work queues by providing a --match string instead of specifying all of the work queues. The agent will poll every work queue with a name that starts with the given string.

Add --match flag to work queues documentation #7768

Add --match flag to work queues documentation #7768

Conversation

riquelmev commented Dec 5, 2022 • edited by zanieb Loading

netlify bot commented Dec 5, 2022 • edited Loading

✅ Deploy Preview for prefect-orion ready!

tpdorsey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zanieb commented Jan 6, 2023

riquelmev commented Dec 5, 2022 •

edited by zanieb

Loading

netlify bot commented Dec 5, 2022 •

edited

Loading