Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: add executor runtime info for explain for connection statement (#19183) #20384

Merged
merged 7 commits into from
Oct 22, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #19183 to release-4.0


What problem does this PR solve?

Issue Number: close #18233

Problem Summary:
add this runtime information in explain for connection statement.

Example:

When set @@tidb_enable_collect_execution_info=1;

test> show processlist;
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| Id  | User | Host      | db   | Command | Time | State      | Info                                                      |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| 775 | root | 127.0.0.1 | test | Query   | 65   | autocommit | select count(*) from t1 tt1 join t1 tt2 where tt1.a>tt2.a |
| 1   | root | 127.0.0.1 | test | Query   | 0    | autocommit | show processlist                                          |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
test> explain for connection 775;
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows | task      | access object         | execution info                                                                                                       | operator info                                             | memory | disk |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0       | root      |                       | time:0s, loops:0                                                                                                     | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 1083392 | root      |                       | time:11.67287184s, loops:1058                                                                                        | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836  | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.729723ms} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0       | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 5120    | root      |                       | time:126.260071ms, loops:5, cop_task: {num: 1, max:125.596953ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.582473ms}   | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0       | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

# after a while
test> explain for connection 775;
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows   | task      | access object         | execution info                                                                   | operator info                                             | memory | disk |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0         | root      |                       | time:0s, loops:0                                                                 | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 145286144 | root      |                       | time:6m3.062011395s, loops:141881                                                | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836    | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0         | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 19456     | root      |                       | time:127.068821ms, loops:19, cop_task: {num: 1, max:125.596953ms, proc_keys: 0}  | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0         | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

When set @@tidb_enable_collect_execution_info=0;, Since TiDB doesn't collect runtime information, so explan for connection will also not display the executor info.

explain for connection 778;
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| id                        | estRows         | task      | access object         | operator info                                             |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| HashAgg_11                | 1.00            | root      |                       | funcs:count(1)->Column#7                                  |
| └─HashJoin_14             | 233130602896.00 | root      |                       | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) |
|   ├─IndexReader_19(Build) | 482836.00       | root      |                       | index:IndexFullScan_18                                    |
|   │ └─IndexFullScan_18    | 482836.00       | cop[tikv] | table:tt2, index:a(a) | keep order:false                                          |
|   └─IndexReader_17(Probe) | 482836.00       | root      |                       | index:IndexFullScan_16                                    |
|     └─IndexFullScan_16    | 482836.00       | cop[tikv] | table:tt1, index:a(a) | keep order:false                                          |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

  • Add executor information in explain for connection result.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

executor/prepared_test.go Outdated Show resolved Hide resolved
@crazycs520
Copy link
Contributor

/run-all-tests

@crazycs520
Copy link
Contributor

/run-all-tests

Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 22, 2020
@SunRunAway
Copy link
Contributor

@qw4990 PTAL

@SunRunAway
Copy link
Contributor

@crazycs520 documentation is needed.

@crazycs520
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 22, 2020
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit fdd7a42 into pingcap:release-4.0 Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution This PR is from a community contributor. sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants