Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: record execute runtime information in slow log (#17573) #18072

Merged
merged 3 commits into from
Jun 18, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #17573 to release-4.0


Signed-off-by: crazycs crazycs520@gmail.com

What problem does this PR solve?

Close #15009

Record execute runtime information in slow log plan, such as below:

  1. Enable collect runtime execute information first:
set tidb_enable_collect_execution_info=1;
  1. Then when TiDB output the slow-log, the Plan field value will like below, It will contain the plan runtime execute information.
# Plan: tidb_decode_plan('2wLwlTAJNl84CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzUwCTEJdGltZTo3NjYuMDk5NzZtcywgbG9vcHM6MiwgUGFydGlhbENvbmN1cnJlbmN5OjQsIEZpbmFsQ29uY3VycmVuY3k6NAkxMC43NDAwMDU0OTMxNjQwNjIgTUIJTi9BCjEJMV8xMQkwCTgwMDAJZXEoQwl7LDMxLCAwKQk3NjE0MBGDIDUuMzU5NjQ4bRWEODc2CTEuOTA5NjAzMTE4OAEdDDQ0IE0JWxgyCTExXzEyAeHwWzAwMDAJdGFibGU6U0xPV19RVUVSWSwgb25seSBzZWFyY2ggaW4gdGhlIGN1cnJlbnQgJ3RpZGItc2xvdy5sb2cnIGZpbGUJNzY0NjkJdGltZTo2NTEuODk5NDAzHY4kNwlOL0EJTi9BCg==')
  1. Decode the plan
test> select  tidb_decode_plan('2wLwlTAJNl84CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzUwCTEJdGltZTo3NjYuMDk5NzZtcywgbG9vcHM6MiwgUGFydGlhbENvbmN1cnJlbmN5OjQsIEZpbmFsQ29uY3VycmVuY3k6NAkxMC43NDAwMDU0OTMxNjQwNjIgTUIJTi9BCjEJMV8xMQkwCTgwMDAJZXEoQwl7LDMxLCAwKQk3NjE0MBGDIDUuMzU5NjQ4bRWEODc2CTEuOTA5NjAzMTE4OAEdDDQ0IE0JWxgyCTExXzEyAeHwWzAwMDAJdGFibGU6U0xPV19RVUVSWSwgb25seSBzZWFyY2gga
   -> W4gdGhlIGN1cnJlbnQgJ3RpZGItc2xvdy5sb2cnIGZpbGUJNzY0NjkJdGltZTo2NTEuODk5NDAzHY4kNwlOL0EJTi9BCg==');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('2wLwlTAJNl84CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzUwCTEJdGltZTo3NjYuMDk5NzZtcywgbG9vcHM6MiwgUGFydGlhbENvbmN1cnJlbmN5OjQsIEZpbmFsQ29uY3VycmVuY3k6NAkxMC43NDAwMDU0OTMxNjQwNjIgTUIJTi9BCjEJMV8xMQkwCTgwMDAJZXEoQwl7LDMxLCAwKQk3NjE0MBGDIDUuMzU5Nj |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|       HashAgg_8               root    1       funcs:count(1)->Column#50                                               1       time:766.09976ms, loops:2, PartialConcurrency:4, FinalConcurrency:4     10.740005493164062 MB   N/A                                                            |
|       └─Selection_11          root    8000    eq(Column#31, 0)                                                        76140   time:765.359648ms, loops:76                                             1.9096031188964844 MB   N/A                                                            |
|         └─MemTableScan_12     root    10000   table:SLOW_QUERY, only search in the current 'tidb-slow.log' file       76469   time:651.899403ms, loops:77                                             N/A                     N/A                                                            |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

As you see, the execute information was contain in the slow-log Plan field now. Since the decode plan field was become too many, we had better add some header column name to more readable, such as below, but we can do this in next PR:

test> select  tidb_decode_plan('2wLwlTAJNl84CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzUwCTEJdGltZTo3NjYuMDk5NzZtcywgbG9vcHM6MiwgUGFydGlhbENvbmN1cnJlbmN5OjQsIEZpbmFsQ29uY3VycmVuY3k6NAkxMC43NDAwMDU0OTMxNjQwNjIgTUIJTi9BCjEJMV8xMQkwCTgwMDAJZXEoQwl7LDMxLCAwKQk3NjE0MBGDIDUuMzU5NjQ4bRWEODc2CTEuOTA5NjAzMTE4OAEdDDQ0IE0JWxgyCTExXzEyAeHwWzAwMDAJdGFibGU6U0xPV19RVUVSWSwgb25seSBzZWFyY2ggaW4gdGhlIGN1cnJlbnQgJ3RpZGItc2xvdy5sb2cnIGZpbGUJNzY0NjkJdGltZ
   -> To2NTEuODk5NDAzHY4kNwlOL0EJTi9BCg==');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('2wLwlTAJNl84CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzUwCTEJdGltZTo3NjYuMDk5NzZtcywgbG9vcHM6MiwgUGFydGlhbENvbmN1cnJlbmN5OjQsIEZpbmFsQ29uY3VycmVuY3k6NAkxMC43NDAwMDU0OTMxNjQwNjIgTUIJTi9BCjEJMV8xMQkwCTgwMDAJZXEoQwl7LDMxLCAwKQk3NjE0MBGDIDUuMzU5Nj |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|       id                      task    estRows operator info                                                           actRows execution info                                                          memory                  disk                                                       |
|       HashAgg_8               root    1       funcs:count(1)->Column#50                                               1       time:766.09976ms, loops:2, PartialConcurrency:4, FinalConcurrency:4     10.740005493164062 MB   N/A                                                        |
|       └─Selection_11          root    8000    eq(Column#31, 0)                                                        76140   time:765.359648ms, loops:76                                             1.9096031188964844 MB   N/A                                                        |
|         └─MemTableScan_12     root    10000   table:SLOW_QUERY, only search in the current 'tidb-slow.log' file       76469   time:651.899403ms, loops:77                                             N/A                     N/A                                                        |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

What is changed and how it works?

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test(ing)
  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • record execute runtime information in slow log.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@crazycs520 please accept the invitation then you can push to the cherry-pick pull requests.
https://github.com/ti-srebot/tidb/invitations

@crazycs520
Copy link
Contributor

wait #18073 merge first.

@crazycs520
Copy link
Contributor

/run-all-tests

@crazycs520
Copy link
Contributor

/run-integration-ddl-test

Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520
Copy link
Contributor

/run-integration-br-test

@zz-jason zz-jason requested review from a team and SunRunAway and removed request for SunRunAway, qw4990, XuHuaiyu, Reminiscent and a team June 17, 2020 08:40
Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qw4990 qw4990 added the status/can-merge Indicates a PR has been approved by a committer. label Jun 18, 2020
@ti-srebot
Copy link
Contributor Author

Sorry @qw4990, you don't have permission to trigger auto merge event on this branch.

@crazycs520
Copy link
Contributor

/run-unit-tests

@crazycs520 crazycs520 merged commit ab8648e into pingcap:release-4.0 Jun 18, 2020
@crazycs520 crazycs520 deleted the release-4.0-36ea109b83c5 branch June 18, 2020 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/config component/expression sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. type/enhancement The issue or PR belongs to an enhancement. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants