Skip to content

Conversation

@peter-toth
Copy link
Contributor

@peter-toth peter-toth commented Jan 21, 2026

What changes were proposed in this pull request?

This is a minor fix of EnsureRequirements to make sure copied scan nodes inherit the tags from original ones in SPJ functions.

Why are the changes needed?

Fix possible bugs.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions
Copy link

JIRA Issue Information

=== Bug SPARK-55113 ===
Summary: EnsureRequirements should copy tags
Assignee: None
Status: Open
Affected: ["4.2.0"]


This comment was automatically generated by GitHub Actions

@peter-toth
Copy link
Contributor Author

cc @dongjoon-hyun

Copy link
Contributor

@allisonwang-db allisonwang-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test if possible?

@peter-toth
Copy link
Contributor Author

peter-toth commented Jan 21, 2026

Can we add a test if possible?

Unfortunately, I don't have any concrete tests that exposes the bug with not copying the tags in populateCommonPartitionInfo() or in populateJoinKeyPositions().
But this issue came up at #53859 (comment) where a very similar populateNoGroupingPartitionInfo() is added and if it doesn't copy the tags a missing logical link tag causes trouble in some AQE tests.

@peter-toth
Copy link
Contributor Author

Thanks for the review! Merged to master (4.2.0).

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @peter-toth and @HyukjinKwon .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants