Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use iterative loop to look for mapped parent #34622

Merged
merged 1 commit into from
Sep 27, 2023

Conversation

uranusjr
Copy link
Member

Alternative fix to #34587.

This changes the logic in views to look for a parent mapped task group (so to decide whether a task group is mapped) from using a recursive function to an iterative loop. This should help prevent RecursionError when a DAG has deeply-nested task groups. I hope no-one really writes such a DAG in practice, but you never know. Also in cases both solutions work, the loop should generally be faster anyway due to how CPython is implemented.

This changes the logic in views to look for a parent mapped task group
(so to decide whether a task group is mapped) from using a recursive
function to an iterative loop. This should help prevent RecursionError
when a DAG has deeply-nested task groups. I hope no-one really writes
such a DAG in practice, but you never know. Also in cases both solutions
work, the loop should generally be faster anyway due to how CPython is
implemented.
Copy link
Member

@hussein-awala hussein-awala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't pay attention that we already have a method to get the mapped task groups (parent tg), LGTM

@uranusjr uranusjr merged commit d9ba152 into apache:main Sep 27, 2023
43 checks passed
@uranusjr uranusjr deleted the find-mapped-group-iterative branch September 27, 2023 00:43
@ephraimbuddy ephraimbuddy added this to the Airflow 2.7.2 milestone Oct 3, 2023
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Oct 3, 2023
ephraimbuddy pushed a commit that referenced this pull request Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:webserver Webserver related Issues type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants