Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

table: use evalBuffer to improve performance of locatePartition (#18818) #19647

Merged
merged 5 commits into from
Sep 5, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18818 to release-3.0


What problem does this PR solve?

Issue Number: close #16667

Problem Summary: MutRowFromDatums is so heavy that cause locatePartition performance worse than expected. Especially in LOAD DATA, partition table need nearly twice time than general table.

What is changed and how it works?

What's Changed:

  • For single column partition key, like partition by hash(col) or partition by range(col), get integer value directly.
  • For expression as partition key, use evalBuffer to avoid MutRowFromDatums repeatly.

How it Works:
This two method will avoid using MutRowFromDatums many times. improve the write performance of partition table.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test (will be added later)
  • Manual test (add detailed scripts or steps below)

Side effects

  • None

Release note

  • table: improve the write performance of partition table

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imtbkcat please resolve conflict and fix CI.

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 3, 2020
@zz-jason
Copy link
Member

zz-jason commented Sep 3, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 3, 2020
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@zz-jason
Copy link
Member

zz-jason commented Sep 4, 2020

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 19689

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@zz-jason
Copy link
Member

zz-jason commented Sep 5, 2020

/merge

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit bd63653 into pingcap:release-3.0 Sep 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/transaction SIG:Transaction status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/enhancement The issue or PR belongs to an enhancement. type/3.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants