fix map_index is null dynamic task mapping bug#54249
Merged
kaxil merged 2 commits intoapache:mainfrom Aug 13, 2025
Merged
Conversation
ba47dd5 to
63c41e0
Compare
63c41e0 to
567e514
Compare
Contributor
o-nikolas
approved these changes
Aug 11, 2025
Contributor
o-nikolas
left a comment
There was a problem hiding this comment.
Changes lgtm, but someone more familiar with task mapping should have a second look.
Lee-W
approved these changes
Aug 13, 2025
Member
Lee-W
left a comment
There was a problem hiding this comment.
Tested with a few different cases and looks good
Backport failed to create: v3-0-test. View the failure log Run details
You can attempt to backport this manually by running: cherry_picker f685c61 v3-0-testThis should apply the commit to the v3-0-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continue |
kaxil
pushed a commit
that referenced
this pull request
Aug 13, 2025
closes: #52881 When tasks within nested task groups try to access XCom values from sibling tasks in the same mapped parent task group instance, the system incorrectly uses `map_index=null` instead of the correct map index, causing XCom lookup failures. <img width="1033" height="494" alt="Screenshot 2025-08-07 at 1 47 06 PM" src="https://github.com/user-attachments/assets/f6c1767b-a999-470d-b090-478643dcdc01" /> ```javascript No XCom value found; defaulting to None.: key="return_value": dag_id="test_dag": task_id="expandable_task_group.inner_task_group.alter_input": run_id="manual__2025-07-04T14:56:47.815002+00:00": map_index=null ``` The issue is in `airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py` in the `_get_upstream_map_indexes` function. The logic only checks the immediate parent task group (upstream_task.task_group) but doesn't consider nested scenarios where tasks are within a regular task group that is itself within a mapped task group. Replace the immediate parent check with a hierarchy-aware check using `get_closest_mapped_task_group()`. This ensures that both tasks recognize they share the same mapped task group ancestor and use the same map_index. - Verified proposed solution resolves issue: #52881 when tested locally using `breeze start-airflow` against the latest main branch development environment. - Added `test_nested_mapped_task_group_upstream_indexes` to verify that tasks in nested mapped task groups correctly resolve upstream map indexes. (cherry picked from commit f685c61)
54 tasks
meldiner
pushed a commit
to faros-ai/airflow
that referenced
this pull request
Feb 6, 2026
closes: apache#52881 When tasks within nested task groups try to access XCom values from sibling tasks in the same mapped parent task group instance, the system incorrectly uses `map_index=null` instead of the correct map index, causing XCom lookup failures. <img width="1033" height="494" alt="Screenshot 2025-08-07 at 1 47 06 PM" src="https://github.com/user-attachments/assets/f6c1767b-a999-470d-b090-478643dcdc01" /> ```javascript No XCom value found; defaulting to None.: key="return_value": dag_id="test_dag": task_id="expandable_task_group.inner_task_group.alter_input": run_id="manual__2025-07-04T14:56:47.815002+00:00": map_index=null ``` The issue is in `airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py` in the `_get_upstream_map_indexes` function. The logic only checks the immediate parent task group (upstream_task.task_group) but doesn't consider nested scenarios where tasks are within a regular task group that is itself within a mapped task group. Replace the immediate parent check with a hierarchy-aware check using `get_closest_mapped_task_group()`. This ensures that both tasks recognize they share the same mapped task group ancestor and use the same map_index. - Verified proposed solution resolves issue: apache#52881 when tested locally using `breeze start-airflow` against the latest main branch development environment. - Added `test_nested_mapped_task_group_upstream_indexes` to verify that tasks in nested mapped task groups correctly resolve upstream map indexes. (cherry picked from commit f685c61)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #52881
Problem Description
When tasks within nested task groups try to access XCom values from sibling tasks in the same mapped parent task group instance, the system incorrectly uses
map_index=nullinstead of the correct map index, causing XCom lookup failures.Example DAG Structure
Error Observed (Note: map_index=null)
Related Code
The issue is in
airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.pyin the_get_upstream_map_indexesfunction.The logic only checks the immediate parent task group (upstream_task.task_group) but doesn't consider nested scenarios where tasks are within a regular task group that is itself within a mapped task group.
Proposed Solution
Replace the immediate parent check with a hierarchy-aware check using
get_closest_mapped_task_group(). This ensures that both tasks recognize they share the same mapped task group ancestor and use the same map_index.Testing
breeze start-airflowagainst the latest main branch development environment.test_nested_mapped_task_group_upstream_indexesto verify that tasks in nested mapped task groups correctly resolve upstream map indexes.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.