Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: add coprocessor cache hit ratio in explain analyze (#19948) #19972

Merged
merged 5 commits into from
Sep 21, 2020

Conversation

ti-srebot
Copy link
Contributor

@ti-srebot ti-srebot commented Sep 14, 2020

cherry-pick #19948 to release-4.0


What problem does this PR solve?

Issue Number: close #17370

What is changed and how it works?

How it Works: add coprocessor cache hit rate in explain analyze.

here is an example:

tidb(localhost:4001) > explain analyze select * from t use index(a);
+-------------------------------+-----------+---------+-----------+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+-----------------------+------+
| id                            | estRows   | actRows | task      | access object          | execution info                                                                                                                                                                                                                                           | operator info                  | memory                | disk |
+-------------------------------+-----------+---------+-----------+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+-----------------------+------+
| IndexLookUp_6                 | 262400.00 | 262400  | root      |                        | time:620.513742ms, loops:258, cop_task: {num: 4, max: 5.530817ms, min: 1.51829ms, avg: 2.70883ms, p95: 5.530817ms, max_proc_keys: 2480, p95_proc_keys: 2480, tot_proc: 1ms, tot_wait: 1ms, rpc_num: 4, rpc_time: 10.816328ms, copr_cache_hit_rate: 0.75} |                                | 6.685169219970703 MB  | N/A  |
| ├─IndexFullScan_4(Build)      | 262400.00 | 262400  | cop[tikv] | table:t, index:a(a, c) | proc max:93ms, min:1ms, p80:93ms, p95:93ms, iters:275, tasks:4                                                                                                                                                                                           | keep order:false, stats:pseudo | 1.7549400329589844 MB | N/A  |
| └─TableRowIDScan_5(Probe)     | 262400.00 | 0       | cop[tikv] | table:t                | time:0ns, loops:0                                                                                                                                                                                                                                        | keep order:false, stats:pseudo | N/A                   | N/A  |
+-------------------------------+-----------+---------+-----------+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+-----------------------+------+
3 rows in set (0.62 sec)

another example with consecutive two same queries.

tidb(localhost:4001) > explain analyze select * from t;
+-----------------------+-----------+---------+-----------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------+--------------------------------+----------------------+------+
| id                    | estRows   | actRows | task      | access object | execution info
                                                                     | operator info                  | memory               | disk |
+-----------------------+-----------+---------+-----------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+----------------------+------+
| TableReader_5         | 262400.00 | 262400  | root      |               | time:398.505526ms, loops:258, cop_task: {num: 3, max: 395.43793ms, min: 126.256914ms, avg: 284.324103ms, p95: 395.43793ms, max_proc_keys: 103120, p95_proc_keys: 103120, tot_proc: 421ms, tot_wait: 11ms, rpc_num: 3, rpc_time: 852.957604ms, copr_cache_hit_rate: 0.00} | data:TableFullScan_4           | 10.83200740814209 MB | N/A  |
| └─TableFullScan_4     | 262400.00 | 262400  | cop[tikv] | table:t       | proc max:159ms, min:41ms, p80:159ms, p95:159ms, iters:269, tasks:3                                                                                                                                                                                                       | keep order:false, stats:pseudo | N/A                  | N/A  |
+-----------------------+-----------+---------+-----------+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+----------------------+------+
2 rows in set (0.42 sec)

tidb(localhost:4001) > explain analyze select * from t;
+-----------------------+-----------+---------+-----------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+-----------------------+------+
| id                    | estRows   | actRows | task      | access object | execution info                                                                                                                                                                | operator info                  | memory                | disk |
+-----------------------+-----------+---------+-----------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------+-----------------------+------+
| TableReader_5         | 262400.00 | 262400  | root      |               | time:8.316905ms, loops:258, cop_task: {num: 3, max: 2.68057ms, min: 2.104203ms, avg: 2.332752ms, p95: 2.68057ms, rpc_num: 3, rpc_time: 6.982763ms, copr_cache_hit_rate: 1.00} | data:TableFullScan_4
           | 10.831978797912598 MB | N/A  |
| └─TableFullScan_4     | 262400.00 | 262400  | cop[tikv] | table:t       | proc max:159ms, min:41ms, p80:159ms, p95:159ms, iters:269, tasks:3                                                                                                            | keep order:false, st
ats:pseudo | N/A                   | N/A  |
+-----------------------+-----------+---------+-----------+---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------
-----------+-----------------------+------+
2 rows in set (0.02 sec)

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU

Release note

  • add coprocessor cache hit ratio in explain analyze.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 14, 2020
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 15, 2020
@SunRunAway
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor Author

Sorry @SunRunAway, you don't have permission to trigger auto merge event on this branch.
The version releasement is in progress.

@SunRunAway
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 21, 2020
@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 20092

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@bb7133
Copy link
Member

bb7133 commented Sep 21, 2020

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 20076
  • 20074
  • 20062
  • 20020

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit 40bb99c into pingcap:release-4.0 Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
challenge-program sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants