-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
executor: support partition pruning for IndexJoin
inner table
#19990
Conversation
/run-common-test |
This solution is not efficient enough, it just narrows down the partitions for a batch of data ( Before the change:
After the change:
The ideal case:
|
eeb4a1d
to
75d185e
Compare
LGTM |
In the current implementation, one table reader is bind to one table, so we have to construct a partition table and change its child -- table reader. |
6658b34
to
583a96b
Compare
LGTM |
/merge |
/run-all-tests |
What problem does this PR solve?
Problem Summary: Although index join support partition table as inner table, it can not do partition prune according to the outer table look up keys. This pr fixes this problem.
What is changed and how it works?
What's Changed:
ColumnOffset
in partition expression informations, which is used for re-construct keys to locate partition.buildPartitionTableForInnerExecutor
which will firstly prune partition according to query conditions and then re-construct partition locate key according to column offset and data from outer table.How it Works:
When extract data from outer table, inner table builder will prune according to outer table data by extract partition keys and eval partition expression. If outer table data doesn't contain enough information, such as lack of partition keys, the pruning will fail.
Related changes
Check List
Tests
Side effects
Release note
IndexJoin
when inner table is partition table.