forked from apache/doris
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Enhancement](Optimizer) Nereids pattern matching base framework (apa…
…che#9474) This pr provide a new pattern matching framework for Nereids optimizer. The new pattern matching framework contains this concepts: 1. `Pattern`/`PatternDescriptor`: the tree node's multiple hierarchy shape, e.g. `logicalJoin(logicalJoin(), any()` pattern describe a plan that root is a `LogicalJoin` and the left child is `LogicalJoin` too. 2. `MatchedAction`: a callback function when the pattern matched, usually you can create new plan to replace the origin matched plan. 3. `MatchingContext`: the param pass through MatchedAction, contains the matched plan root and the PlannerContext. 4. `PatternMatcher`: contains PatternDescriptor and MatchedAction 5. `Rule`: a rewrite rule contains RuleType, PatternPromise, Pattern and transform function(equals to MatchedAction) 6. `RuleFactory`: the factory can help us build Rules easily. RuleFactory extends Patterns interface, and have some predefined pattern descriptors. for example, Join commutative: ```java public class JoinCommutative extends OneExplorationRuleFactory { @OverRide public Rule<Plan> build() { return innerLogicalJoin().thenApply(ctx -> { return new LogicalJoin( JoinType.INNER_JOIN, ctx.root.getOnClause(), ctx.root.right(), ctx.root.left() ); }).toRule(RuleType.LOGICAL_JOIN_COMMUTATIVE); } } ``` the code above show the three step to create a Rule 1. 'innerLogicalJoin()' declare pattern is an inner logical join. 'innerLogicalJoin' is a predefined pattern. 2. invoke 'thenApply()' function to combine a MatchedAction, return a new LogicalJoin with exchange children. 3. invoke 'toRule()' function to convert to Rule You can think the Rule contains three parts: 1. Pattern 2. transform function / MatchedAction 3. RuleType and RulePromise So 1. `innerLogicalJoin()` create a `PatternDescriptor`, which contains a `Pattern` 2. `PatternDescriptor.then()` convert `PatternDescriptor` to `PatternMatcher,` witch contains Pattern and MatchedAction 3. `PatternMatcher.toRule()` convert `PatternMatcher` to a Rule This three step inspired by the currying in function programing. It should be noted, apache#9446 provide a generic type for TreeNode's children, so we can infer multiple hierarchy type in this pattern matching framework, so you can get the really tree node type without unsafely cast. like this: ```java logicalJoin(logicalJoin(), any()).then(j -> { // j can be inferred type to LogicalJoin<LogicalJoin<Plan, Plan>, Plan> // so j.left() can be inferred type to LogicalJoin<Plan, Plan>, // so you don't need to cast j.left() from 'Plan' to 'LogicalJoin' var node = j.left().left(); }) ```
- Loading branch information
Showing
56 changed files
with
1,235 additions
and
191 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
28 changes: 28 additions & 0 deletions
28
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/MatchedAction.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| package org.apache.doris.nereids.pattern; | ||
|
|
||
| import org.apache.doris.nereids.trees.TreeNode; | ||
|
|
||
| /** | ||
| * Define an callback action when match a pattern, usually implement as a rule body. | ||
| * e.g. exchange join children for JoinCommutative Rule | ||
| */ | ||
| public interface MatchedAction<INPUT_TYPE extends TreeNode, OUTPUT_TYPE extends TreeNode> { | ||
| OUTPUT_TYPE apply(MatchingContext<INPUT_TYPE> ctx); | ||
| } |
43 changes: 43 additions & 0 deletions
43
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/MatchingContext.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| package org.apache.doris.nereids.pattern; | ||
|
|
||
| import org.apache.doris.nereids.PlannerContext; | ||
| import org.apache.doris.nereids.trees.TreeNode; | ||
|
|
||
| /** | ||
| * Define a context when match a pattern pass through a MatchedAction. | ||
| */ | ||
| public class MatchingContext<T extends TreeNode> { | ||
| public final T root; | ||
| public final Pattern<T> pattern; | ||
| public final PlannerContext plannerContext; | ||
|
|
||
| /** | ||
| * the MatchingContext is the param pass through the MatchedAction. | ||
| * | ||
| * @param root the matched tree node root | ||
| * @param pattern the defined pattern | ||
| * @param plannerContext the planner context | ||
| */ | ||
| public MatchingContext(T root, Pattern<T> pattern, PlannerContext plannerContext) { | ||
| this.root = root; | ||
| this.pattern = pattern; | ||
| this.plannerContext = plannerContext; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
62 changes: 62 additions & 0 deletions
62
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/PatternDescriptor.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| package org.apache.doris.nereids.pattern; | ||
|
|
||
| import org.apache.doris.nereids.rules.RulePromise; | ||
| import org.apache.doris.nereids.trees.TreeNode; | ||
|
|
||
| import java.util.ArrayList; | ||
| import java.util.List; | ||
| import java.util.Objects; | ||
| import java.util.function.Function; | ||
| import java.util.function.Predicate; | ||
|
|
||
| /** | ||
| * Define a descriptor to wrap a pattern tree to define a pattern shape. | ||
| * It can support pattern generic type to MatchedAction. | ||
| */ | ||
| public class PatternDescriptor<INPUT_TYPE extends RULE_TYPE, RULE_TYPE extends TreeNode> { | ||
| public final Pattern<INPUT_TYPE> pattern; | ||
| public final RulePromise defaultPromise; | ||
| public final List<Predicate<INPUT_TYPE>> predicates = new ArrayList<>(); | ||
|
|
||
| public PatternDescriptor(Pattern<INPUT_TYPE> pattern, RulePromise defaultPromise) { | ||
| this.pattern = Objects.requireNonNull(pattern, "pattern can not be null"); | ||
| this.defaultPromise = Objects.requireNonNull(defaultPromise, "defaultPromise can not be null"); | ||
| } | ||
|
|
||
| public PatternDescriptor<INPUT_TYPE, RULE_TYPE> when(Predicate<INPUT_TYPE> predicate) { | ||
| predicates.add(predicate); | ||
| return this; | ||
| } | ||
|
|
||
| public <OUTPUT_TYPE extends RULE_TYPE> PatternMatcher<INPUT_TYPE, OUTPUT_TYPE, RULE_TYPE> then( | ||
| Function<INPUT_TYPE, OUTPUT_TYPE> matchedAction) { | ||
| return new PatternMatcher<>(patternWithPredicates(), defaultPromise, ctx -> matchedAction.apply(ctx.root)); | ||
| } | ||
|
|
||
| public <OUTPUT_TYPE extends RULE_TYPE> PatternMatcher<INPUT_TYPE, OUTPUT_TYPE, RULE_TYPE> thenApply( | ||
| MatchedAction<INPUT_TYPE, OUTPUT_TYPE> matchedAction) { | ||
| return new PatternMatcher<>(patternWithPredicates(), defaultPromise, matchedAction); | ||
| } | ||
|
|
||
| public Pattern<INPUT_TYPE> patternWithPredicates() { | ||
| Pattern[] children = pattern.children().toArray(new Pattern[0]); | ||
| return new Pattern<>(pattern.getNodeType(), predicates, children); | ||
| } | ||
| } |
Oops, something went wrong.