Skip to content

Add ordering to AssetEvent query in SchedulerJobRunner#52231

Merged
kaxil merged 3 commits intoapache:mainfrom
dingo4dev:fix/select_asset_events_with_order
Aug 13, 2025
Merged

Add ordering to AssetEvent query in SchedulerJobRunner#52231
kaxil merged 3 commits intoapache:mainfrom
dingo4dev:fix/select_asset_events_with_order

Conversation

@dingo4dev
Copy link
Contributor

@dingo4dev dingo4dev commented Jun 25, 2025

closes: #52230

This change is adding ordering for selecting asset events to maintain the ordering.

@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Jun 25, 2025
@dingo4dev dingo4dev force-pushed the fix/select_asset_events_with_order branch 2 times, most recently from 8d69d27 to eb33a83 Compare June 25, 2025 09:43
@dingo4dev dingo4dev marked this pull request as ready for review June 25, 2025 10:08
Copilot AI review requested due to automatic review settings June 25, 2025 10:08
@dingo4dev dingo4dev requested review from XD-DENG and ashb as code owners June 25, 2025 10:08
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds deterministic ordering to the AssetEvent query in SchedulerJobRunner so that events are processed chronologically.

  • Introduces an .order_by(AssetEvent.timestamp.asc()) clause on the AssetEvent select.
  • Ensures asset events fetched in ascending timestamp order.
Comments suppressed due to low confidence (1)

airflow-core/src/airflow/jobs/scheduler_job_runner.py:1603

  • Add or update a unit test to verify that asset events are returned in ascending timestamp order, ensuring the new ordering clause works as intended.
                .order_by(AssetEvent.timestamp.asc())

@dingo4dev dingo4dev changed the title [WIP] Add ordering to AssetEvent query in SchedulerJobRunner Add ordering to AssetEvent query in SchedulerJobRunner Jun 25, 2025
@dingo4dev dingo4dev force-pushed the fix/select_asset_events_with_order branch from eb33a83 to dc6e838 Compare June 25, 2025 15:20
@dingo4dev dingo4dev force-pushed the fix/select_asset_events_with_order branch from dc6e838 to f25b682 Compare June 26, 2025 01:32
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@dingo4dev
Copy link
Contributor Author

@ashb @XD-DENG TBR.

@eladkal eladkal requested a review from kaxil July 30, 2025 18:40
@eladkal eladkal added this to the Airflow 3.0.4 milestone Jul 30, 2025
@eladkal eladkal added type:bug-fix Changelog: Bug Fixes backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch labels Jul 30, 2025
@eladkal eladkal modified the milestones: Airflow 3.0.4, Airflow 3.0.5 Aug 8, 2025
@kaxil kaxil requested review from Lee-W and uranusjr August 13, 2025 12:34
@kaxil kaxil merged commit c8336ec into apache:main Aug 13, 2025
59 checks passed
@github-actions
Copy link

Backport failed to create: v3-0-test. View the failure log Run details

Status Branch Result
v3-0-test Commit Link

You can attempt to backport this manually by running:

cherry_picker c8336ec v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

kaxil pushed a commit that referenced this pull request Aug 13, 2025
closes: #52230

This change is adding ordering for selecting asset events to maintain the ordering.

(cherry picked from commit c8336ec)
@dingo4dev dingo4dev deleted the fix/select_asset_events_with_order branch August 13, 2025 14:27
meldiner pushed a commit to faros-ai/airflow that referenced this pull request Feb 6, 2026
closes: apache#52230

This change is adding ordering for selecting asset events to maintain the ordering.

(cherry picked from commit c8336ec)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Scheduler including HA (high availability) scheduler backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect ordering of events in triggering_asset_events

4 participants

Comments