-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature](Nereids) Support materialized index selection. #13416
Conversation
1f3196a
to
edf566b
Compare
@@ -543,7 +398,7 @@ public CheckContext(LogicalOlapScan scan) { | |||
) | |||
)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Map<Boolean, Map<String, Column>> nameToColumnGroupingByIsKey
= scan.getTable().getSchemaByIndexId(indexId)
.collect(Collectors.groupingBy(
Column::isKey,
Collectors.toMap(Column::getName, Function.identity())
));
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rename checkHasNoValueTypeColumn
method to disablePreAggIfContainsAnyValueColumn
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
@@ -313,22 +312,13 @@ public PlanFragment visitPhysicalOlapScan(PhysicalOlapScan olapScan, PlanTransla | |||
tupleDescriptor.setRef(tableRef); | |||
olapScanNode.setSelectedPartitionIds(olapScan.getSelectedPartitionIds()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
olapScanNode should be immutable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
olapScanNode
is the plan node in old planner. OlapScanNode.setXXX
is to set the result of nereids planner to OlapScanNode
in old planner.
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
we should flat filter candidate indexes to refine code later. |
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Count.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java
Outdated
Show resolved
Hide resolved
...src/main/java/org/apache/doris/nereids/rules/mv/SelectMaterializedIndexWithoutAggregate.java
Outdated
Show resolved
Hide resolved
*/ | ||
public boolean shouldSelectRollup() { | ||
public boolean shouldSelectIndex() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to remove this function and use isIndexSelected instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was to do two things:
- judge
indexSelected
is true or false - make sure the keys type of the OLAP table is
DUP_KEYS
,UNIQUE_KEYS
, orAGG_KEYS
, but it should not bePRIMARY_KEYS
. Even if it seemsPRIMARY_KEYS
is not used, it's safe to ensure the scope to apply materialized view selection rules is right.
I would keep the getter method of the status of indexSelected
in LogicalOlapScan
and move the condition of applying materialized view selection to the rule itself.
Proposed changes
This PR unified the selection of rollup index and materialized view index into uniform logic, which is called selecting materialized index.
Main steps:
Find candidate indexes
SelectMaterializedIndexWithAggregate
. The base index and indexes that could use pre-aggregation should be used. The pre-aggregation status is determined by aggregation function, grouping expression, and pushdown predicates.SelectMaterializedIndexWithoutAggregate
. The base index and indexes that have all the key columns could be used.Filter and order the candidate indexes
Problem summary
Describe your changes.
Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...