planner: open fully order prop push down for partition table's table scan and single index scan #42694

winoros · 2023-03-29T17:33:56Z

What problem does this PR solve?

Issue Number: ref #26166

Problem Summary:

What is changed and how it works?

We've implemented merge operation between partitions of table scan and single index scan.
So we don't need the former quick path.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

…scan and single index scan

ti-chi-bot · 2023-03-29T17:33:58Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

Defined2014
qw4990

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

planner/core/casetest/integration_test.go

…t-push-down-for-part-table

qw4990 · 2023-04-26T02:45:47Z

/test mysql-test

qw4990 · 2023-04-26T02:45:53Z

/merge

ti-chi-bot · 2023-04-26T02:45:57Z

This pull request has been accepted and is ready to merge.

Commit hash: 9ca5616

winoros · 2023-04-26T08:29:42Z

/merge

ti-chi-bot · 2023-04-26T08:29:46Z

This pull request has been accepted and is ready to merge.

Commit hash: 8b0b953

time-and-fate · 2023-04-26T11:14:51Z

planner/core/find_best_task.go

+			if ds.tableInfo.GetPartitionInfo() != nil && !is.Index.Global {
+				is.Columns = append(is.Columns, model.NewExtraPhysTblIDColInfo())
+				is.schema.Append(&expression.Column{
+					RetType:  types.NewFieldType(mysql.TypeLonglong),
+					UniqueID: ds.ctx.GetSessionVars().AllocPlanColumnID(),
+					ID:       model.ExtraPhysTblID,
+				})
+			}


Why do we need this? Does this conflict with (*PhysicalIndexScan).initSchema()?

No, that case is to add a partition id for the global index(It's decoded from the value of the global index). Here we add an extra partition id for non-global index which is decoded from the key instead of the value.

ExtraPhysTblID will let TiKV return PID in chunk. IndexWorker will use the pid to create a partitionHandle for tableWorker found the handle belongs to which PID and use it to keep order.

time-and-fate · 2023-04-26T11:37:13Z

planner/core/task.go

 	} else {
 		return nil, false
 	}


This entire if-else branch can be simplified.

This func will be removed in the next pr since there's fully support for IndexMerge

winoros · 2023-04-26T12:08:42Z

/merge

ti-chi-bot · 2023-04-26T12:08:47Z

This pull request has been accepted and is ready to merge.

Commit hash: 218a241

winoros · 2023-04-26T12:18:12Z

/cherry-pick release-7.1

ti-chi-bot · 2023-04-26T12:18:54Z

@winoros: new pull request created to branch release-7.1: #43441.

In response to this:

/cherry-pick release-7.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

planner: open fully order prop push down for partition table's table …

7723846

…scan and single index scan

ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 29, 2023

winoros added 2 commits March 30, 2023 01:46

tiny change and fix test

54460d5

fix test

50cb177

winoros force-pushed the open-some-full-limit-push-down-for-part-table branch from 64d068a to 50cb177 Compare March 29, 2023 18:29

Defined2014 reviewed Mar 31, 2023

View reviewed changes

planner/core/casetest/integration_test.go Show resolved Hide resolved

Defined2014 approved these changes Mar 31, 2023

View reviewed changes

planner/core/casetest/integration_test.go Show resolved Hide resolved

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 31, 2023

winoros added 2 commits April 4, 2023 19:31

Merge remote-tracking branch 'origin/master' into open-some-full-limi…

70bdf68

…t-push-down-for-part-table

add index lookup

6857844

winoros force-pushed the open-some-full-limit-push-down-for-part-table branch from 5517030 to 6857844 Compare April 4, 2023 12:32

fix more tests

00bb701

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 4, 2023

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 14, 2023

Merge branch 'master' into open-some-full-limit-push-down-for-part-table

9ca5616

ti-chi-bot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 25, 2023

ti-chi-bot bot deleted a comment from ti-chi-bot Apr 25, 2023

qw4990 approved these changes Apr 26, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 26, 2023

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 26, 2023

qw4990 added the sig/planner SIG: Planner label Apr 26, 2023

ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 26, 2023

Merge branch 'master' into open-some-full-limit-push-down-for-part-table

64dae3e

ti-chi-bot bot removed status/can-merge Indicates a PR has been approved by a committer. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 26, 2023

fix the merge error

8b0b953

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 26, 2023

fix merge error again

84cfe5d

ti-chi-bot bot removed the status/can-merge Indicates a PR has been approved by a committer. label Apr 26, 2023

time-and-fate reviewed Apr 26, 2023

View reviewed changes

fix merge error again*2

218a241

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 26, 2023

ti-chi-bot bot merged commit 4b6f0ce into pingcap:master Apr 26, 2023

winoros deleted the open-some-full-limit-push-down-for-part-table branch April 26, 2023 12:10

ti-chi-bot mentioned this pull request Apr 26, 2023

planner: open fully order prop push down for partition table's table scan and single index scan (#42694) #43441

Closed

12 tasks

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 26, 2023

This is an automated cherry-pick of pingcap#42694

fe9d1a0

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

XuHuaiyu mentioned this pull request Dec 15, 2023

v7.1.3, NONCLUSTED INDEX, hash partition table, report "Internal error: UnionExec chunk column count mismatch, req: 3, result: 4" #49509

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: open fully order prop push down for partition table's table scan and single index scan #42694

planner: open fully order prop push down for partition table's table scan and single index scan #42694

winoros commented Mar 29, 2023

ti-chi-bot commented Mar 29, 2023 •

edited by ti-chi-bot bot

Loading

qw4990 commented Apr 26, 2023

qw4990 commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

winoros commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

time-and-fate Apr 26, 2023 •

edited

Loading

winoros Apr 26, 2023

Defined2014 Apr 26, 2023 •

edited

Loading

time-and-fate Apr 26, 2023

winoros Apr 26, 2023

winoros commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

winoros commented Apr 26, 2023

ti-chi-bot commented Apr 26, 2023

planner: open fully order prop push down for partition table's table scan and single index scan #42694

planner: open fully order prop push down for partition table's table scan and single index scan #42694

Conversation

winoros commented Mar 29, 2023

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Mar 29, 2023 • edited by ti-chi-bot bot Loading

qw4990 commented Apr 26, 2023

qw4990 commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

winoros commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

time-and-fate Apr 26, 2023 • edited Loading

Choose a reason for hiding this comment

winoros Apr 26, 2023

Choose a reason for hiding this comment

Defined2014 Apr 26, 2023 • edited Loading

Choose a reason for hiding this comment

time-and-fate Apr 26, 2023

Choose a reason for hiding this comment

winoros Apr 26, 2023

Choose a reason for hiding this comment

winoros commented Apr 26, 2023

ti-chi-bot bot commented Apr 26, 2023

winoros commented Apr 26, 2023

ti-chi-bot commented Apr 26, 2023

ti-chi-bot commented Mar 29, 2023 •

edited by ti-chi-bot bot

Loading

time-and-fate Apr 26, 2023 •

edited

Loading

Defined2014 Apr 26, 2023 •

edited

Loading