[release/6.0] Query: Match correct predicate structure to convert apply to join #27292
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
smitpatel
changed the title
maumar
approved these changes
Jan 27, 2022
|
@smitpatel could you resolve the conflicts here? |
Resolves #26756 For property access on optional dependents sharing column with principal, we generate CaseExpression. In order to match them during join key search, we updated our recursive function to match shape of the test in CaseExpression which incorrectly matched similar structure outside of case block causing somewhat wrong join key to be extracted. While join key in itself could work, when generating partitions out of it due to paging operation, it gives incorrect results. The fix is to match the special structure of CaseExpression.Test separately. Also made the key comparison match more robust by only allowing column or case block to appear in condition. Other structures don't represent part of join key in a comparison. Null checks for join keys are handled separately so that we only remove null checks which are indeed used in comparison with other columns in other operations.
smitpatel
force-pushed
the
smit/6.0/issue26756
branch
from
17b2756 to
804137a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #26756
For property access on optional dependents sharing column with principal, we generate CaseExpression. In order to match them during join key search, we updated our recursive function to match shape of the test in CaseExpression which incorrectly matched similar structure outside of case block causing somewhat wrong join key to be extracted. While join key in itself could work, when generating partitions out of it due to paging operation, it gives incorrect results.
The fix is to match the special structure of CaseExpression. Test separately. Also made the key comparison match more robust by only allowing column or case block to appear in condition. Other structures don't represent part of join key in a comparison.
Null checks for join keys are handled separately so that we only remove null checks which are indeed used in comparison with other columns in other operations.
Description
When a correlated subquery which has paging applied to it is converted to join, we detect incorrect join predicate causing wrong partitions to be created for row number operation which gives wrong result.
Customer impact
Customers running query with affected scenarios can receive incorrect results.
How found
Customer reported on 6.0
Regression
Yes. From 5.0. We fixed bug in 6.0 to generate a complex expression for optional dependent. To match this new structure we updated join predicate detection code which inadvertently matched wrong thing.
Testing
Added test for user reported scenario. Some existing tests which had wrong SQL also got updated. The tests for original fix in 6.0 are also passing.
Risk
Low risk. We are matching what is exactly required now. Also added quirk to revert to previous behavior.