Skip to content

Conversation

@guan404ming
Copy link
Member

@guan404ming guan404ming commented Sep 22, 2025

Add number of db queries guard in list endpoint, preventing further N+1 queries problem

N+1 queries problem detected and solved.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@guan404ming guan404ming force-pushed the fix-lazy-loading branch 2 times, most recently from 3c5d0fd to cf5aa7d Compare October 30, 2025 09:16
@guan404ming guan404ming marked this pull request as ready for review October 30, 2025 10:29
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am working on this for all 'list' endpoint, across all entities and public/ui apis.

I am first taking care of only list endpoint because those are the most critical for N+1 query problem. Then we can do the same for other endpoints.

#57508

You can probably rebase on the PR above, keep the xcom part and the 'noload'. Rename the PR description and add the assert_queries_count in tests. Too

@guan404ming
Copy link
Member Author

guan404ming commented Oct 30, 2025

I am first taking care of only list endpoint because those are the most critical for N+1 query problem. Then we can do the same for other endpoints.

Sure, I've modified this PR to ensure there is no N+1 query issue for list xcom endpoints first.

@guan404ming guan404ming changed the title Set lazy loading default to 'noload' for alias case Add number of queries guard in public xcom entries list endpoints Oct 30, 2025
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@pierrejeambrun pierrejeambrun added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Oct 30, 2025
@pierrejeambrun pierrejeambrun added this to the Airflow 3.1.2 milestone Oct 30, 2025
@pierrejeambrun pierrejeambrun merged commit fc72417 into apache:main Oct 30, 2025
115 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 30, 2025
…ndpoints (#55936)

(cherry picked from commit fc72417)

Co-authored-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
@github-actions
Copy link

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Oct 30, 2025
…ndpoints (apache#55936)

(cherry picked from commit fc72417)

Co-authored-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
pierrejeambrun pushed a commit that referenced this pull request Oct 30, 2025
…ndpoints (#55936) (#57554)

(cherry picked from commit fc72417)

Co-authored-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Nov 10, 2025
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants