Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flow runs from deployments #7047

Merged
merged 50 commits into from
Oct 6, 2022
Merged

Flow runs from deployments #7047

merged 50 commits into from
Oct 6, 2022

Conversation

anticorrelator
Copy link
Contributor

@anticorrelator anticorrelator commented Sep 30, 2022

Adds a utility to create flow runs from a deployment and waits for a specified amount of time before returning the FlowRun object. Users can both override the default Deployment parameters as well as specify unlimited (or no) polling before returning from this function.

Closes #6850

Example

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • This pull request includes tests or only affects documentation.
  • This pull request includes a label categorizing the change e.g. fix, feature, enhancement

@netlify
Copy link

netlify bot commented Sep 30, 2022

Deploy Preview for prefect-orion ready!

Name Link
🔨 Latest commit f90fe51
🔍 Latest deploy log https://app.netlify.com/sites/prefect-orion/deploys/633e4dd10a501b0009a928fb
😎 Deploy Preview https://deploy-preview-7047--prefect-orion.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Copy link
Contributor

@anna-geller anna-geller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QAed and it works well 🎉

One thing to consider is that currently, both run_deployment and schedule_deployment result in no auto-scheduled flag being set, some might expect that schedule_deployment leads to auto_scheduled=True. The current behavior seems correct to me since those don't run on a regular cadence (the schedule_deployment is technically a single ad-hoc run but just not executed immediately) but this is a matter of interpretation

src/prefect/coordination.py Outdated Show resolved Hide resolved
Copy link
Member

@cicdw cicdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few minor notes, this LGTM otherwise!

src/prefect/client/utilities.py Outdated Show resolved Hide resolved
src/prefect/client/utilities.py Outdated Show resolved Hide resolved
src/prefect/orion/models/deployments.py Show resolved Hide resolved
anticorrelator and others added 2 commits October 5, 2022 18:49
Co-authored-by: Chris White <chris@prefect.io>
Copy link
Contributor

@anna-geller anna-geller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great!

for docs - code used for QA

from prefect.client.utilities import run_deployment

# run parametrized flow from deployment and wait for its completion before starting the next task
flow_run_obj = run_deployment(
    "parametrized/dev",
    parameters=dict(user="QA Run - waiting"),
)

# run parametrized flow from deployment and wait for its completion before starting the next task
run_deployment(
    "parametrized/dev",
    max_polls=-1,
    parameters=dict(user="QA Run - waiting forever until completion - no poll limits"),
)
# run parametrized flow from deployment and don't wait for completion
run_deployment(
    "parametrized/dev",
    max_polls=0,
    parameters=dict(user="QA Run - fire-and-forget 🔥"),
)

# run parametrized flow from deployment and don't wait for completion
not_finished_flow_run = run_deployment(
    "parametrized/dev",
    max_polls=2,
    poll_interval=2,
    parameters=dict(user="QA Run - very short polling"),
)
print(not_finished_flow_run.state)  # Pending

Copy link
Contributor

@abrookins abrookins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Copy link
Contributor

@zanieb zanieb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exciting! Mostly asking for a change around timeout handling.

@zanieb
Copy link
Contributor

zanieb commented Oct 6, 2022

Thanks for working through so many people's feedback!

@zanieb zanieb merged commit 94b9e76 into main Oct 6, 2022
@zanieb zanieb deleted the single-deployment-runs branch October 6, 2022 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose a utility that creates a flow run from a deployment
5 participants