Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: pessimistic lock on the temporary table should not be written to TiKV #24737

Merged
merged 6 commits into from
May 26, 2021

Conversation

tiancaiamao
Copy link
Contributor

@tiancaiamao tiancaiamao commented May 19, 2021

What problem does this PR solve?

Avoid any kind of (pessimistic) lock on the temporary table been written to TiKV

Problem Summary:

What is changed and how it works?

The concept of "lock" is complex.

  1. select ... for update

Basically, it means lock for the read operation.
We use MVCC and the transaction isolation level is actually "snapshot isolation", there are locks for the write operation but no locks for the read operation.
"write skew" is problem in "snapshot isolation". To overcome it, we provide "select ... for update" which also lock for read and can be used to fix the "write skew" problem.

  1. pessimistic / optimistic

From TiDB 3.0, we start to support the pessimistic transaction mode.
Pessimistic transaction differs from the optimistic transaction that it write locks to the TiKV immediately, instead of caching the changes in memory and write to TiKV until commit.

  1. (batch) point get / coprocessor

(batch) point get is a special code path for performance optimization.
With two different implementation, we have to take special care to maintain the different code path.
And for the pessimistic transaction, there are some minor behaviour differences between point get / non-point get request.

Under any circumstance, operations on the temporary table should not write locks to the TiKV.

In #24540 we cover the point get and batch point get.
However, there are 2*2*2 = 8 combinations in total, to cover them all, I chose the common code path.

What's Changed:

Filter out the keys belong to the temporary table in doLockKeys(), all the pessimistic mode lock goes through this code path.

How it Works:

All the optimistic mode lock should have already been filtered before the 2PC commit in PR #24196.

So all the cases are covered after this PR.

Check List

Tests

  • Unit test

Release note

  • No release note

@tiancaiamao tiancaiamao requested a review from a team as a code owner May 19, 2021 03:34
@tiancaiamao tiancaiamao requested review from lzmhhh123 and removed request for a team May 19, 2021 03:34
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label May 19, 2021
@tiancaiamao tiancaiamao mentioned this pull request May 19, 2021
89 tasks
@tiancaiamao tiancaiamao requested a review from coocood May 19, 2021 03:41
@github-actions github-actions bot added the sig/execution SIG execution label May 19, 2021
@coocood
Copy link
Member

coocood commented May 19, 2021

can we revert #24540 now?

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 19, 2021
@tiancaiamao
Copy link
Contributor Author

can we revert #24540 now?

I think that would hurt our contributor's feeling or at least discourage their enthusiasm for contribution.
#24540 is not introducing a bug or something wrong, it's just not solve all the cases in one PR. @coocood

@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 19, 2021
@coocood
Copy link
Member

coocood commented May 19, 2021

can we revert #24540 now?

I think that would hurt our contributor's feeling or at least discourage their enthusiasm for contribution.
#24540 is not introducing a bug or something wrong, it's just not solve all the cases in one PR. @coocood

Or just remove the code that doing duplicated calculation to save CPU?

@tiancaiamao tiancaiamao requested a review from a team as a code owner May 25, 2021 08:04
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 25, 2021
@tiancaiamao
Copy link
Contributor Author

#24540 is reverted, PTAL @coocood

@tiancaiamao
Copy link
Contributor Author

/run-check_dev
/run-check_dev_2

@coocood
Copy link
Member

coocood commented May 25, 2021

/lgtm

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 25, 2021
@tiancaiamao tiancaiamao requested review from lysu and removed request for a team May 25, 2021 09:39
@lysu
Copy link
Contributor

lysu commented May 26, 2021

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • coocood
  • lysu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 26, 2021
@lysu
Copy link
Contributor

lysu commented May 26, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 6364a82

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 26, 2021
@ti-chi-bot
Copy link
Member

@tiancaiamao: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot merged commit f79dc8b into pingcap:master May 26, 2021
@tiancaiamao tiancaiamao deleted the temporary-pessimistic-lock branch June 17, 2021 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants