Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: add pessimistic lock keys runtime information (#19547) #20199

Merged
merged 7 commits into from
Oct 10, 2020
Merged

executor: add pessimistic lock keys runtime information (#19547) #20199

merged 7 commits into from
Oct 10, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #19547 to release-4.0

What problem does this PR solve?

In pessimistic transaction, the statement maybe needs to lock keys, this PR add lock keys runtime information in executor runtime stats. Here is an example:

test> select tidb_decode_plan('8wTgMAkyOF81CTAJMAlOL0EJMAl0aW1lOjE0NS40MTA5MjVtcywgbG9vcHM6MSwgbG9ja19rZXlzOiB7BSggMzQuMDU2OTM4ASc0cmVnaW9uOjEwMjQsIGsBJhAyMDAyOA03NHJwYzozMjEuNzE4NzYxBTAccGNfY291bnQFMzR9CTI4LjQ1MzEyNSBLQgGNJAoxCTI2XzcJMAkBSGA5LjIyNTg5NTMxNjgwNglmb3IgdXBkYXRlBR4AOAm4IDUwLjUzNzgzNAFgAGwFtwQyMQFMAQQcCjIJMzFfMTBWUQA8ZGF0YTpTZWxlY3Rpb25fOS5XACAxMi42NzU5NDYuVwBELCBjb3BfdGFzazoge251bTogCfosbWF4OiA0LjI3N
   -> zc4BTFkbWluOiAyMy4xMsK1cywgYXZnOiA3Ny40NjYFECxwOTU6IDEyNS4xMTEFESh0b3RfcHJvYzogOQFwIScdXwEPJXxkIDc2LjkyNzY0NG1zfQkxNS4xOTQzMzU5Mzc1QRwzCTFfOQkxX1JCAUxsdCh0ZXN0LnQxLmEsIDQwMDAwKVEADDBucywtQEkYJE4vQQo0CTEwXzgJTzA1MjUxCXRhYmxlOnQxIe40ZXAgb3JkZXI6ZmFsc2VySAA=');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('8wTgMAkyOF81CTAJMAlOL0EJMAl0aW1lOjE0NS40MTA5MjVtcywgbG9vcHM6MSwgbG9ja19rZXlzOiB7BSggMzQuMDU2OTM4ASc0cmVnaW9uOjEwMjQsIGsBJhAyMDAyOA03NHJwYzozMjEuNzE4NzYxBTAccGNfY291bnQFMzR9CTI4LjQ1MzEyNSBLQgGNJAoxCTI2XzcJMAkBSGA5LjIyNTg5NTMxNjgwNglmb3IgdX                                           |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     id                     task         estRows               operator info                 actRows    execution info                                                                                                                                                             memory              disk |
|     Update_5               root         0                     N/A                           0          time:145.410925ms, loops:1, lock_keys: {time:34.056938ms, region:1024, keys:20028, lock_rpc:321.718761ms, rpc_count:1024}                                                  28.453125 KB        N/A  |
|     └─SelectLock_7         root         20029.225895316806    for update                    20028      time:50.537834ms, loops:21                                                                                                                                                 N/A                 N/A  |
|       └─TableReader_10     root         20029.225895316806    data:Selection_9              20028      time:12.675946ms, loops:21, cop_task: {num: 1024, max: 4.277786ms, min: 23.12µs, avg: 77.466µs, p95: 125.111µs, tot_proc: 9ms, rpc_num: 1024, rpc_time: 76.927644ms}    15.1943359375 KB    N/A     |
|         └─Selection_9      cop[tikv]    20029.225895316806    lt(test.t1.a, 40000)          0          time:0ns, loops:0                                                                                                                                                          N/A                 N/A  |
|           └─TableScan_8    cop[tikv]    25251                 table:t1, keep order:false    0          time:0ns, loops:0                                                                                                                                                          N/A                 N/A  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
test> select tidb_decode_plan('4ATwdTAJNl85CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzQJMQl0aW1lOjUwLjgxNzg0MW1zLCBsb29wczoyLCBQYXJ0aWFsQ29uY3VycmVuY3k6NSwgRmluYWxDb25jdXJyZW5jeTo1LCBsb2NrX2tleXM6IHsFUSAyNS42OTc4MzYBUVhyZWdpb246MTAyNCwga2V5czoyMDAyOAFqQGNrX3JwYzoxNTQuNDM5NTA5BTAIcGNfBawFM2x9CTE4OS42ODc1IEtCCU4vQQoxCTI2XzEyCTAJAUhgOS4yMjU4OTUzMTY4MDYJZm9yIHVwZGF0ZQUeBDgJBZAYNTAuNzI0OTLhAAAxAU0BBBwKMgkzMl8xNFZRABhpbmRle
   -> DpJAQYsUmFuZ2VTY2FuXzEzLl4AHDE4LjcxNzc3Be4EbG8lP4gxLCBjb3BfdGFzazoge251bTogMSwgbWF4OjExLjE4NDU0NQHsDHByb2MtOggwLCAB+hErAQwFvAAgBTFMMTk4NW1zfQkyODEuMzA2NjQwNjI1EgwzCTEzAZQEMV9SFAEkdGFibGU6dDEsIAnNGGEoYSksIHIBzjA6Wy1pbmYsNDAwMDApIZ48ZXAgb3JkZXI6ZmFsc2UJMClAEDBucywgCdokMAlOL0EJTi9BCg==');
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('4ATwdTAJNl85CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzQJMQl0aW1lOjUwLjgxNzg0MW1zLCBsb29wczoyLCBQYXJ0aWFsQ29uY3VycmVuY3k6NSwgRmluYWxDb25jdXJyZW5jeTo1LCBsb2NrX2tleXM6IHsFUSAyNS42OTc4MzYBUVhyZWdpb246MTAyNCwga2V5czoyMDAyOAFqQGNrX3JwYzoxNTQuNDM5NT                                                                     |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     id                    task         estRows               operator info                                                 actRows    execution info                                                                                                                                                        memory              disk |
|     HashAgg_9             root         1                     funcs:count(1)->Column#4                                      1          time:50.817841ms, loops:2, PartialConcurrency:5, FinalConcurrency:5, lock_keys: {time:25.697836ms, region:1024, keys:20028, lock_rpc:154.439509ms, rpc_count:1024}    189.6875 KB         N/A  |
|     └─SelectLock_12       root         20029.225895316806    for update                                                    20028      time:50.724941ms, loops:21                                                                                                                                            N/A                 N/A  |
|       └─IndexReader_14    root         20029.225895316806    index:IndexRangeScan_13                                       20028      time:18.717776ms, loops:21, cop_task: {num: 1, max:11.184545ms, proc_keys: 0, rpc_num: 1, rpc_time: 11.181985ms}                                                      281.306640625 KB    N/A  |
|         └─IndexScan_13    cop[tikv]    20029.225895316806    table:t1, index:a(a), range:[-inf,40000), keep order:false    0          time:0ns, loops:0                                                                                                                                                     N/A                 N/A  |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

What is changed and how it works?

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • Add pessimistic lock keys runtime information

@crazycs520 @ti-srebot
cherry pick pingcap#19547 to release-4.0
9c355f9 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot mentioned this pull request Sep 24, 2020
Merged
@ti-srebot ti-srebot added this to the v4.0.8 milestone Sep 24, 2020
@crazycs520
Revert "cherry pick pingcap#19547 to release-4.0"
6d72b72 
This reverts commit 9c355f9.
@crazycs520
Merge branch 'release-4.0' of https://github.com/pingcap/tidb into re…
56e6fdc 
…lease-4.0-915d84d7ad4a
@crazycs520
executor: add pessimistic lock keys runtime information (pingcap#19547)
abe6478 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
fix test
69195e2 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
fix collect stats
c80887f 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
Copy link
Contributor

/run-all-tests

@crazycs520
Copy link
Contributor

/run-unit-test

wjhuang2016
wjhuang2016 reviewed Oct 9, 2020
View reviewed changes
Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor Author

@wjhuang2016, Thanks for your review, however we are sorry that your vote won't be count.

qw4990
qw4990 reviewed Oct 9, 2020
View reviewed changes
Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 9, 2020
@crazycs520
Copy link
Contributor

/run-unit-test

@crazycs520 crazycs520 mentioned this pull request Oct 9, 2020
Merged
@crazycs520
Copy link
Contributor

/run-unit-test

@crazycs520
Copy link
Contributor

/run-all-tests

@zz-jason
Copy link
Member

zz-jason commented Oct 9, 2020

/merge

@ti-srebot
Copy link
Contributor Author

@zz-jason Oops! This PR requires at least 2 LGTMs to merge. The current number of LGTM is 1

@wshwsh12 wshwsh12 removed their request for review October 9, 2020 15:13
SunRunAway
SunRunAway reviewed Oct 10, 2020
View reviewed changes
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 10, 2020
@crazycs520
Copy link
Contributor

/run-integration-copr-test
/run-unit-test

@SunRunAway SunRunAway merged commit 08c63bf into pingcap:release-4.0 Oct 10, 2020
@SunRunAway SunRunAway deleted the release-4.0-915d84d7ad4a branch October 10, 2020 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunRunAway SunRunAway SunRunAway left review comments

@wjhuang2016 wjhuang2016 wjhuang2016 left review comments

@qw4990 qw4990 qw4990 approved these changes

Assignees

@crazycs520 crazycs520

Labels
component/executor sig/execution SIG execution status/LGT2 Indicates that a PR has LGTM 2. type/4.0-cherry-pick
Projects
None yet
Milestone
v4.0.8
Development

Successfully merging this pull request may close these issues.
6 participants
@ti-srebot @crazycs520 @zz-jason @SunRunAway @qw4990 @wjhuang2016