Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: Introduce PessimisticRRTxnContextProvider for pessimistic repeatable read txn #35158

Merged
merged 21 commits into from
Jun 13, 2022

Conversation

SpadeA-Tang
Copy link
Contributor

@SpadeA-Tang SpadeA-Tang commented Jun 6, 2022

Signed-off-by: SpadeA-Tang u6748471@anu.edu.au

What problem does this PR solve?

Issue Number: close #35129

What is changed and how it works?

  • Provider transaction context provider for pessimistic repeatable read isolation level.
  • Add interface TxnAdvisable which is implemented by TxnManager and TxnContextProvider. It providers a unified entrance for providering various optimizations. Now, it provides AdviseWarmup (which fetches ts from PD asynchronously) and AdviseOptimizeWithPlan (which decide to not use latest ts for point get for update related executation to reduce latency).
  • Implemente error handling within provider.
  • Add unit tests.

todo:

  • Now, there are some places acquiring ts from TransactionContext. I will open another PR to refactor them.

Check List

Tests

  • Unit test

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jun 6, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • ekexium
  • lcwangchao

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 6, 2022
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
@sre-bot
Copy link
Contributor

sre-bot commented Jun 6, 2022

Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
executor/builder.go Outdated Show resolved Hide resolved
sessiontxn/isolation/readcommitted.go Outdated Show resolved Hide resolved
sessiontxn/isolation/repeatable_read.go Show resolved Hide resolved
sessiontxn/isolation/repeatable_read.go Outdated Show resolved Hide resolved
}

mayOptimizeForPointGet := false
if v, ok := plan.(*plannercore.PhysicalLock); ok {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some cases can also use this optimization, for example:

select v from t where id=1 and v>1 for update; -- Projection <- SelectLock <- Selection <- PointGet'
select * from t where id=1 union all select * from t where id=2; -- Union <- SelectLock <- PointGet
select * from t where id=1 union select * from t where id=1; -- HashAgg <- Union  <- SelectLock <- PointGet

I think BatchPointGet can also use this optimization, but the old code did not do it, any problem if we do it now?
@cfzjywxk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it's ok as the locking behaviors are similar. Actually different behaviors for batch/point get have introduced us much trouble for us just like #32045.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BatchPointGet does not use txnManager.GetStmtForUpdateTS() in executorBuilder.getSnapshotTS(), which means it will not acquire ts from PD.

session/txnmanager.go Outdated Show resolved Hide resolved
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
planner/optimize.go Outdated Show resolved Hide resolved
@@ -120,7 +122,7 @@ type TxnContextProvider interface {
// OnStmtRetry is the hook that should be called when a statement is retried internally.
OnStmtRetry(ctx context.Context) error
// Advise is used to give advice to provider
Advise(tp AdviceType) error
Advise(tp AdviceType, val []any) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to avoid using any type but a concrete type in the interface definition, it would make it difficult to use and maintain.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard to construct a concrete type here as the parameters of different advice type differs a lot. Could you give some suggestions?

Copy link
Contributor

@cfzjywxk cfzjywxk Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on the design of what's planned to be passed inside and how the parameters would be used. The most commonly used way is to abstract some interface or trait to describe the meaning of the input parameters, for example, you could abstract a new type named TxnAdvice which contains both advice type and value, or abstract a new interface which defines what should be passed in as the advice parameters.
The main thing is to answer what's an advice and how to describe it, what's the design?, but not just leave something like whatever type is acceptable, perhaps some will work some would not especially defining interfaces.

In general, don't use the any or inteface{} types it's not maintainable and other people have no idea what should be passed or how to build another advice type.

sessiontxn/isolation/base.go Outdated Show resolved Hide resolved
// Because stale read will use `staleread.StalenessTxnContextProvider` for query, so if `staleread.IsStmtStaleness()`
// returns true in other providers, it means the current statement is `BeginStmt` with stale read
func (p *baseTxnContextProvider) isBeginStmtWithStaleRead() bool {
return staleread.IsStmtStaleness(p.sctx)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's the logic checking isBeiginStmt, the function name is a bit confusing.

sessiontxn/isolation/repeatable_read.go Outdated Show resolved Hide resolved
sessiontxn/isolation/repeatable_read.go Outdated Show resolved Hide resolved
}

mayOptimizeForPointGet := false
if v, ok := plan.(*plannercore.PhysicalLock); ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems it's ok as the locking behaviors are similar. Actually different behaviors for batch/point get have introduced us much trouble for us just like #32045.

Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
@cfzjywxk cfzjywxk requested a review from TonsnakeLin June 8, 2022 12:15
@ekexium
Copy link
Member

ekexium commented Jun 9, 2022

It seems to me some part of this change has overlaps/conflicts with #35131? Is there a dependency between the two?

session/session.go Outdated Show resolved Hide resolved
@SpadeA-Tang
Copy link
Contributor Author

It seems to me some part of this change has overlaps/conflicts with #35131? Is there a dependency between the two?

These two PRs will be unified eventually. The later one may need to solve the merge conflictss.

Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
@SpadeA-Tang
Copy link
Contributor Author

/run-unit-test

SpadeA-Tang and others added 3 commits June 10, 2022 16:28
Co-authored-by: ekexium <ekexium@fastmail.com>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Copy link
Member

@ekexium ekexium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 10, 2022
SpadeA-Tang and others added 3 commits June 10, 2022 17:01
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 13, 2022
@lcwangchao
Copy link
Collaborator

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: f2ee63f

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 13, 2022
@ti-chi-bot ti-chi-bot merged commit 738f681 into pingcap:master Jun 13, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Jun 13, 2022

TiDB MergeCI notify

🔴 Bad News! [1] CI still failing after this pr merged.
These failed integration tests don't seem to be introduced by the current PR.

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 2, success 9, total 11 16 min Existing failure
idc-jenkins-ci/integration-cdc-test 🟢 all 35 tests passed 29 min Existing passed
idc-jenkins-ci-tidb/common-test 🟢 all 12 tests passed 10 min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 6 min 51 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 6 min 43 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 6 min 35 sec Existing passed
idc-jenkins-ci-tidb/integration-ddl-test 🟢 all 6 tests passed 5 min 53 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 3 min 10 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 1 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement provider for RR
7 participants