Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: avoid distsql request for TableReader/IndexReader/IndexLookup on temporary table #24769

Merged
merged 16 commits into from
Jun 2, 2021

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

Problem Summary:

For the global temporary table, we use UnionScan on an empty table to read the data.
The temporary table data are actually stored in the transaction's membuffer.
Before this PR, although the result is always empty, we still send the distsql requests to the TiKV.
This PR is an optimization to avoid that.

What is changed and how it works?

What's Changed:

Do not send distsql request for TableReader/IndexReader/IndexLookup on temporary table

How it Works:

After the kv range is calculated, return immediately, without sending distsql request or fetching the result.

Check List

Tests

  • Unit test

Release note

  • No release note

@tiancaiamao tiancaiamao requested a review from a team as a code owner May 19, 2021 14:40
@tiancaiamao tiancaiamao requested review from qw4990 and removed request for a team May 19, 2021 14:40
@ti-chi-bot ti-chi-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 19, 2021
@tiancaiamao tiancaiamao mentioned this pull request May 19, 2021
89 tasks
@tiancaiamao tiancaiamao requested a review from djshow832 May 19, 2021 14:40
@github-actions github-actions bot added the sig/execution SIG execution label May 19, 2021
@tiancaiamao
Copy link
Contributor Author

PTAL @djshow832 @qw4990

@@ -196,6 +196,10 @@ type IndexReaderExecutor struct {

// Close clears all resources hold by current object.
func (e *IndexReaderExecutor) Close() error {
if e.table != nil && e.table.Meta().TempTableType != model.TempTableNone {
Copy link
Contributor

@qw4990 qw4990 May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about implementing a new function in table/table.go for the logic about checking whether the table is a TmpTable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we do not need to check e.table != nil.
It's special here because a test case construct the IndexReaderExecutor manually and don't set e.table field.

The remain part is e.table.Meta().TempTableType != model.TempTableNone.
Define a one line function for that does not seems beneficial: the change will introduce a new exported function without simplifying the code much.

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost LGTM, please address comments.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 25, 2021
@tisonkun
Copy link
Contributor

tisonkun commented Jun 1, 2021

Please refer the pull request to its issue #24169 and it would be better if we create a dedicated sub-issue for this pr.

Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • crazycs520
  • qw4990

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 2, 2021
@tiancaiamao
Copy link
Contributor Author

/merge

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 2, 2021
@tiancaiamao
Copy link
Contributor Author

[2021-06-02T06:22:41.346Z] WARNING: DATA RACE
[2021-06-02T06:22:41.346Z] Read at 0x00c025dd7778 by goroutine 376:
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor.(*recordSet).Next()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/adapter.go:153 +0x31d
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/session.(*execStmtResult).Next()
[2021-06-02T06:22:41.346Z]       <autogenerated>:1 +0x84
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/session.GetRows4Test()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/tidb.go:307 +0x35e
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/session.ResultSetToStringSlice()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/tidb.go:325 +0xba
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor_test.testSerialSuite.TestTemporaryTableNoNetwork.func2()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor_test.go:8383 +0x9a
[2021-06-02T06:22:41.346Z] 
[2021-06-02T06:22:41.346Z] Previous write at 0x00c025dd7778 by goroutine 256:
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor.ResetContextOfStmt()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor.go:1787 +0xef2
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/session.(*session).ExecuteStmt()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1504 +0x204
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).Exec()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:170 +0x2fc
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).MustQuery()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:324 +0x219
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).HasPlan()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:225 +0xb3
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor_test.testSerialSuite.TestTemporaryTableNoNetwork()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor_test.go:8396 +0x644
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.parseTerm()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/terms.go:148 +0x256
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.parse()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/terms.go:125 +0xb3
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.newTerms()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/terms.go:97 +0x6f
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.(*Failpoint).Enable()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/failpoint.go:53 +0x53
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.(*Failpoints).Enable()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/failpoints.go:104 +0x171
[2021-06-02T06:22:41.346Z]   github.com/pingcap/failpoint.Enable()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/failpoint@v0.0.0-20210316064728-7acb0f0a3dfd/failpoints.go:224 +0x2b5
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor_test.testSerialSuite.TestTemporaryTableNoNetwork()
[2021-06-02T06:22:41.346Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor_test.go:8371 +0x2b6
[2021-06-02T06:22:41.346Z]   github.com/pingcap/tidb/executor_test.(*testSerialSuite).TestTemporaryTableNoNetwork()
[2021-06-02T06:22:41.346Z]       <autogenerated>:1 +0x66
[2021-06-02T06:22:41.346Z]   runtime.call16()
[2021-06-02T06:22:41.346Z]       /usr/local/go/src/runtime/asm_amd64.s:550 +0x3d
[2021-06-02T06:22:41.346Z]   reflect.Value.Call()
[2021-06-02T06:22:41.346Z]       /usr/local/go/src/reflect/value.go:337 +0xd8
[2021-06-02T06:22:41.346Z]   github.com/pingcap/check.(*suiteRunner).forkTest.func1()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xb3b
[2021-06-02T06:22:41.346Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()
[2021-06-02T06:22:41.346Z]       /nfs/cache/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x11d

A data race is detected... I need to fix it.

@tiancaiamao
Copy link
Contributor Author

/merge cancel

@ti-chi-bot ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jun 2, 2021
@tiancaiamao
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: daaf042

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 2, 2021
@tiancaiamao
Copy link
Contributor Author

/run-integration-ddl-test

@ti-chi-bot ti-chi-bot merged commit 8656b5d into pingcap:master Jun 2, 2021
@tiancaiamao tiancaiamao deleted the temporary-no-network branch June 17, 2021 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants