Aip 86: add deadlines to DagRunResponse #50957

rawwar · 2025-05-22T12:21:56Z

…line_to_dagrun

bbovenzi · 2025-05-22T17:09:12Z

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py

    end_date: datetime | None
    data_interval_start: datetime | None
    data_interval_end: datetime | None
+    deadlines: list[DeadlineResponse] | None


Oh I missed this. A dag run can have multiple deadlines?

Yes, here's an example that @ferruzzi provided:

with DAG( dag_id="dag_with_deadlines", deadline=[ DeadlineAlert( reference = DeadlineReference.DAGRUN_QUEUED_AT, interval=timedelta(hours=1), callback=warn_me_it_may_be_late ), DeadlineAlert( reference = DeadlineReference.DAGRUN_QUEUED_AT, interval=timedelta(hours=2), callback=definitely_late ), DeadlineAlert( reference = DeadlineReference.DAGRUN_QUEUED_AT, interval=timedelta(hours=3), callback=email_my_boss_that_it_wasnt_my_fault ), ] )

Ok and then on the DagResponse we will have the full DeadlineAlert with the reference, interval and callback?

How about making this a separate endpoint and calling it for the DagRun retrieved? Simply passing the dag_run_id. This way, we don't increase the DagRunResponse too much. It already has a lot of information. We can have deadline routes as a separate if it is okay from the UI perspective

I think keeping some deadline information on DagRunResponse is valuable because I think "show me recent dag runs with passed deadlines" would be a key feature.

From the API point of view, we can nest the deadline API inside the dagrun one. Because deadline can be considered a sub-resource of the dagrun, they only exist for their corresponding dagrun, and especially when the dagrun is in a specific state, otherwise they are removed. (But that's debatable, since all the API resource are for now flat at root level, we could do the same here).

deadlines: list[DeadlineResponse] should be enough, the relationship should return an empty list when there is no related items

I brought it up on the dev call last week and we're going to make (a variation on) the change you suggested. Here's what the lifespan of a Deadline will be, using Dagrun as an example:

When a new Dagrun is created:

If the dag has a deadline, calculate the value and store it along with the dag_id, run_id, etc in the deadline table

When a dagrun finishes:

If (and only if) the current time is before the calculated deadline, then remove the deadline from the deadline table

The scheduler loop will query the deadline table to see if any deadlines have expired

If yes then the callback is sent to the Triggerer to process

once the callback is run, the Triggerer will move the failed deadline to a new table (working name is just missed_deadlines?)

So deadlines table will have all upcoming and unprocessed deadlines. missed_deadlines table will have all expired deadlines which have been resolved. If the serialized DAG has a deadline and it is in neither table, then it will be assumed that it completed successfully.

I should also clarify that the current code as written is still a single deadline, but I need to make that change before launch, it WILL accept a list but the current code in the repo doesn't show that. Sorry if I mislead or confused anyone on that point. It came up in a Meetup discussion after I did the initial work, but it's a great idea and I'm going to be adding it in.

@ferruzzi , if that's the case, should i keep this PR on hold until we add the support to have multiple deadlines?

pierrejeambrun

Nice, looks good to me.

Just a few suggestions.

pierrejeambrun · 2025-06-02T07:57:46Z

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py

    end_date: datetime | None
    data_interval_start: datetime | None
    data_interval_end: datetime | None
+    deadlines: list[DeadlineResponse] | None


Maybe I misunderstood something. As I understand it, there's no record in the db at all for deadlines unless the run is in progress?

I assume we can always improve later and add that if people request it. (soft delete etc...)

pierrejeambrun · 2025-06-02T08:01:05Z

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py

    end_date: datetime | None
    data_interval_start: datetime | None
    data_interval_end: datetime | None
+    deadlines: list[DeadlineResponse] | None


From the API point of view, we can nest the deadline API inside the dagrun one. Because deadline can be considered a sub-resource of the dagrun, they only exist for their corresponding dagrun, and especially when the dagrun is in a specific state, otherwise they are removed. (But that's debatable, since all the API resource are for now flat at root level, we could do the same here).

pierrejeambrun · 2025-06-02T08:05:38Z

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py

    end_date: datetime | None
    data_interval_start: datetime | None
    data_interval_end: datetime | None
+    deadlines: list[DeadlineResponse] | None


deadlines: list[DeadlineResponse] should be enough, the relationship should return an empty list when there is no related items

pierrejeambrun · 2025-06-02T08:06:02Z

airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_run.py

+def convert_instrumented_list_to_dict(lst):
+    result = [obj.__dict__ for obj in lst]
+    for obj in result:
+        obj.pop("_sa_instance_state", None)
+    return result
+
+


Is that needed ?

Yes. That's an additional item on the list and was failing tests due to a mismatch in comparison caused by _sa_instance_state.

…line_to_dagrun

bugraoz93

Changes look good. Some tests still need updates to include the new field in dag_run

…line_to_dagrun

pierrejeambrun

FYI, this PR was merged recently #51698 which will impact this PR.

rawwar · 2025-06-17T18:04:24Z

FYI, this PR was merged recently #51698 which will impact this PR.

@pierrejeambrun , what kind of impact? Did you mean merge conflicts or do I need to be updating something?

pierrejeambrun

@pierrejeambrun , what kind of impact? Did you mean merge conflicts or do I need to be updating something?

Yes merge conflicts and also some unused code. It basically introduces DeadlineAlertResponse which is exactly your DeadlineResponse, so we need to keep only one.

rawwar · 2025-06-24T04:49:26Z

I haven't been getting time to finish this. I'll let someone pick and finish this off. At least the PR can act as a starting point to anyone who picks it up

rawwar added 8 commits May 22, 2025 00:09

Add Deadline relationship to DagRun and update Deadline model

b9c6156

Refactor Deadline relationship in DagRun and Deadline models

980233f

add test

e65e392

call it deadlines

b07c470

remove test

e9a4c07

Merge branch 'main' of github.com:apache/airflow into AIP-86/add_dead…

74e28dc

…line_to_dagrun

add deadlines to DAGRunResponse

5270ebf

update id to be uuid and add tests

f1d3cf8

boring-cyborg bot added area:airflow-ctl area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels May 22, 2025

fix dag run tests

a14ec65

rawwar marked this pull request as ready for review May 22, 2025 15:11

rawwar requested review from bbovenzi, bugraoz93, ephraimbuddy, jason810496, jscheffl, kaxil, pierrejeambrun, potiuk, ryanahamilton and shubhamraj-git as code owners May 22, 2025 15:11

fix tests

6b692c3

rawwar requested a review from ferruzzi May 22, 2025 16:43

bbovenzi reviewed May 22, 2025

View reviewed changes

ambika-garg mentioned this pull request May 25, 2025

Update dag endpoint tests to include dags with deadlines #50992

Open

2 tasks

pierrejeambrun reviewed Jun 2, 2025

View reviewed changes

Merge branch 'main' of github.com:apache/airflow into AIP-86/add_dead…

6c2524c

…line_to_dagrun

bugraoz93 reviewed Jun 15, 2025

View reviewed changes

rawwar added 2 commits June 16, 2025 12:15

Merge branch 'main' of github.com:apache/airflow into AIP-86/add_dead…

25275a7

…line_to_dagrun

update deadline to deadline_time

52afcb3

pierrejeambrun reviewed Jun 16, 2025

View reviewed changes

ferruzzi mentioned this pull request Jun 16, 2025

AIP-86: Add deadline to DagResponse #51698

Merged

fix tests

13a0162

fix tests

92761db

rawwar requested review from bugraoz93 and pierrejeambrun June 17, 2025 18:32

pierrejeambrun reviewed Jun 19, 2025

View reviewed changes

rawwar closed this Jun 24, 2025

Aip 86: add deadlines to DagRunResponse #50957

Aip 86: add deadlines to DagRunResponse #50957

Uh oh!

Conversation

rawwar commented May 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rawwar May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bugraoz93 May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bugraoz93 left a comment

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

rawwar commented Jun 17, 2025

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

rawwar commented Jun 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rawwar May 22, 2025 •

edited

Loading

bugraoz93 May 22, 2025 •

edited

Loading