Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: unify OR type IndexMerge code paths #58396

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
3188f0b
refactor
time-and-fate Dec 9, 2024
4401a67
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 9, 2024
68cf5ef
fix lint
time-and-fate Dec 9, 2024
41e24b9
update test
time-and-fate Dec 9, 2024
e45d7a2
merge 2 mv index OR code paths into 1
time-and-fate Dec 10, 2024
ac0378d
switch the last path to the new implementation
time-and-fate Dec 10, 2024
a138caf
unify into single code path
time-and-fate Dec 10, 2024
d72e454
update test result
time-and-fate Dec 10, 2024
d6fe22a
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 10, 2024
62b0180
fix fix control case and update test result
time-and-fate Dec 10, 2024
f30186c
fix missing index filters
time-and-fate Dec 10, 2024
37e4f39
restore test result
time-and-fate Dec 10, 2024
e62949b
update test result
time-and-fate Dec 10, 2024
ce19969
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 11, 2024
a431242
remove unneeded code
time-and-fate Dec 11, 2024
33b1cba
refactor and add comments 1
time-and-fate Dec 13, 2024
63151a8
fix lint
time-and-fate Dec 13, 2024
c38a0fe
refactor comments
time-and-fate Dec 16, 2024
7291678
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 16, 2024
472b51c
refactor
time-and-fate Dec 16, 2024
c8bd4e5
refactor
time-and-fate Dec 16, 2024
b843a52
refactor
time-and-fate Dec 16, 2024
a2d0042
add comments
time-and-fate Dec 16, 2024
5cec70a
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 16, 2024
cb25d66
add comments and small simplification
time-and-fate Dec 18, 2024
b80ab93
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 18, 2024
3819d1c
add tests
time-and-fate Dec 19, 2024
4832dcb
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 19, 2024
8534dd0
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 22, 2024
45dc306
update test result
time-and-fate Dec 22, 2024
3d3b12e
revert wrong modification
time-and-fate Dec 22, 2024
d2b2d74
fix missing TableFilters
time-and-fate Dec 22, 2024
339fdfa
fix TableFilters for partial table path
time-and-fate Dec 23, 2024
de0b471
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 23, 2024
11c7464
add tests for `SliceRecursiveFlattenIter()`
time-and-fate Dec 23, 2024
f758eae
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 23, 2024
5d1f9de
update BUILD.bazel
time-and-fate Dec 23, 2024
d586e9b
update BUILD.bazel
time-and-fate Dec 23, 2024
53aaeda
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 23, 2024
3c0c9bd
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 25, 2024
dd3d870
minor improve `sliceRecursiveFlattenIterHelper` and add comments
time-and-fate Dec 25, 2024
34b8f95
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 25, 2024
31af6c6
update BUILD.bazel and rename variable
time-and-fate Dec 25, 2024
4634e58
Merge remote-tracking branch 'upstream/master' into s27-indexmerge-no…
time-and-fate Dec 25, 2024
e18ead4
update
time-and-fate Dec 25, 2024
9da5944
format comments
time-and-fate Dec 26, 2024
39e4e7a
add the previous pruning logic
time-and-fate Dec 26, 2024
3bed336
fix import order
time-and-fate Dec 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions pkg/planner/cardinality/testdata/cardinality_suite_out.json
Original file line number Diff line number Diff line change
Expand Up @@ -1483,12 +1483,24 @@
"expr": "((a < 5)) or ((a > 10 and true))",
"row_count": 6
},
{
"table_name": "t",
"type": "Index Stats-Range",
"expr": "((a < 5))",
"row_count": 6
},
{
"table_name": "t",
"type": "Index Stats-Range",
"expr": "((a < 5)) or ((a > 10 and true))",
"row_count": 6
},
{
"table_name": "t",
"type": "Index Stats-Range",
"expr": "((a > 10 and true))",
"row_count": 1
},
{
"table_name": "t",
"type": "Table Stats-Expression-CNF",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,12 +128,12 @@
"Plan": [
"PartitionUnion 33.00 root ",
"├─IndexMerge 11.00 root type: union",
"│ ├─TableRangeScan(Build) 1.00 cop[tikv] table:t, partition:p0 range:[1,1], keep order:false, stats:pseudo",
"│ ├─IndexRangeScan(Build) 10.00 cop[tikv] table:t, partition:p0, index:b(b) range:[2,2], keep order:false, stats:pseudo",
"│ ├─TableRangeScan(Build) 1.00 cop[tikv] table:t, partition:p0 range:[1,1], keep order:false, stats:pseudo",
"│ └─TableRowIDScan(Probe) 11.00 cop[tikv] table:t, partition:p0 keep order:false, stats:pseudo",
"├─IndexMerge 11.00 root type: union",
"│ ├─TableRangeScan(Build) 1.00 cop[tikv] table:t, partition:p1 range:[1,1], keep order:false, stats:pseudo",
"│ ├─IndexRangeScan(Build) 10.00 cop[tikv] table:t, partition:p1, index:b(b) range:[2,2], keep order:false, stats:pseudo",
"│ ├─TableRangeScan(Build) 1.00 cop[tikv] table:t, partition:p1 range:[1,1], keep order:false, stats:pseudo",
"│ └─TableRowIDScan(Probe) 11.00 cop[tikv] table:t, partition:p1 keep order:false, stats:pseudo",
"└─TableReader 11.00 root MppVersion: 2, data:ExchangeSender",
" └─ExchangeSender 11.00 mpp[tiflash] ExchangeType: PassThrough",
Expand Down
23 changes: 1 addition & 22 deletions pkg/planner/core/find_best_task.go
Original file line number Diff line number Diff line change
Expand Up @@ -945,7 +945,7 @@ func matchPropForIndexMergeAlternatives(ds *logicalop.DataSource, path *util.Acc
// if matchIdxes greater than 1, we should sort this match alternative path by its CountAfterAccess.
alternatives := oneORBranch
slices.SortStableFunc(matchIdxes, func(a, b int) int {
res := cmpAlternativesByRowCount(alternatives[a], alternatives[b])
res := cmpAlternatives(ds.SCtx().GetSessionVars())(alternatives[a], alternatives[b])
if res != 0 {
return res
}
Expand Down Expand Up @@ -1033,27 +1033,6 @@ func matchPropForIndexMergeAlternatives(ds *logicalop.DataSource, path *util.Acc
break
}
}
pushDownCtx := util.GetPushDownCtx(ds.SCtx())
for _, path := range determinedIndexPartialPaths {
// If any partial path contains table filters, we need to keep the whole DNF filter in the Selection.
if len(path.TableFilters) > 0 {
if !expression.CanExprsPushDown(pushDownCtx, path.TableFilters, kv.TiKV) {
// if this table filters can't be pushed down, all of them should be kept in the table side, cleaning the lookup side here.
path.TableFilters = nil
}
shouldKeepCurrentFilter = true
}
// If any partial path's index filter cannot be pushed to TiKV, we should keep the whole DNF filter.
if len(path.IndexFilters) != 0 && !expression.CanExprsPushDown(pushDownCtx, path.IndexFilters, kv.TiKV) {
shouldKeepCurrentFilter = true
// Clear IndexFilter, the whole filter will be put in indexMergePath.TableFilters.
path.IndexFilters = nil
}
}
// Keep this filter as a part of table filters for safety if it has any parameter.
if expression.MaybeOverOptimized4PlanCache(ds.SCtx().GetExprCtx(), []expression.Expression{path.IndexMergeORSourceFilter}) {
shouldKeepCurrentFilter = true
}
if shouldKeepCurrentFilter {
// add the cnf expression back as table filer.
indexMergePath.TableFilters = append(indexMergePath.TableFilters, path.IndexMergeORSourceFilter)
Expand Down
Loading