Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dry run for backfill #45062

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Conversation

prabhusneha
Copy link
Contributor

Closes #44395

Response:
image

@boring-cyborg boring-cyborg bot added the area:UI Related to UI/UX. For Frontend Developers. label Dec 18, 2024
@phanikumv phanikumv requested a review from dstandish December 19, 2024 13:59
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with daniel, maybe a separate endpoint makes more sense to avoid mixed returned type BackfillResponse | BackfillDryRunResponse on the same endpoint. That's hard to handle for clients.

class BackfillRunInfo(BaseModel):
"""Data model for run information during a backfill operation."""

logical_date: datetime
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will only have logical date, no data interval?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the logical date is a datetime, so for example for DAGs with schedules other than daily, the response will still include the dates on which the backfill will be triggered. I imagine that the user likely has an understanding of the dag's schedule when making a backfill request. Does including the data interval provide additional value in this context?

image

)
backfill_response_item = []
print(info_list)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to remove print statements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:UI Related to UI/UX. For Frontend Developers.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add robust dry run capability for backfill
3 participants