Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infoschema: add plan field to the statement summary tables (#14182) #14285

Merged
merged 13 commits into from
Jan 2, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented Dec 30, 2019

cherry-pick #14182 to release-3.0

conflicting files:
statement_summary.go(undeclared function)
tables_test.go(import error)


What problem does this PR solve?

Add plan and plan digest to the statement summary tables.
Same SQLs with different plans are summarized in different records in the tables.
Record the SQL and plan that appear at the first time in each summary, not the last time, to increase performance.

Since same SQL may be split into several records, max-stmt-count should increase.

What is changed and how it works?

Add plan_digest and plan fields.
schema_name + digest + prev_sql_digest + plan_digest are combined as a key of the summary map.
Default value of max-stmt-count is increased to 200.
Record the SQL and plan that appear at the first time in each summary, not the last time.

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)
mysql> select * from performance_schema.events_statements_summary_by_digest where plan != ''\G
*************************** 1. row ***************************
       SUMMARY_BEGIN_TIME: 2019-12-22 16:00:00
         SUMMARY_END_TIME: 2019-12-22 16:30:00
                STMT_TYPE: select
              SCHEMA_NAME:
                   DIGEST: ff1ba2b3cf4f7452291642f6399447979825a310ef25a78f01288580f398a3c9
              DIGEST_TEXT: select table_id , is_index , hist_id , distinct_count , version , null_count , tot_col_size , stats_ver , flag , correlation , last_analyze_pos from mysql . stats_histograms where table_id = ?
              TABLE_NAMES: mysql.stats_histograms
              INDEX_NAMES: stats_histograms:tbl
              SAMPLE_USER: NULL
               EXEC_COUNT: 17
              SUM_LATENCY: 17842186
              MAX_LATENCY: 1417064
              MIN_LATENCY: 697887
              AVG_LATENCY: 1049540
        AVG_PARSE_LATENCY: 91016
        MAX_PARSE_LATENCY: 150406
      AVG_COMPILE_LATENCY: 309173
      MAX_COMPILE_LATENCY: 389786
             COP_TASK_NUM: 34
     AVG_COP_PROCESS_TIME: 0
     MAX_COP_PROCESS_TIME: 0
  MAX_COP_PROCESS_ADDRESS: NULL
        AVG_COP_WAIT_TIME: 0
        MAX_COP_WAIT_TIME: 0
     MAX_COP_WAIT_ADDRESS: NULL
         AVG_PROCESS_TIME: 0
         MAX_PROCESS_TIME: 0
            AVG_WAIT_TIME: 0
            MAX_WAIT_TIME: 0
         AVG_BACKOFF_TIME: 0
         MAX_BACKOFF_TIME: 0
           AVG_TOTAL_KEYS: 0
           MAX_TOTAL_KEYS: 0
       AVG_PROCESSED_KEYS: 0
       MAX_PROCESSED_KEYS: 0
        AVG_PREWRITE_TIME: 0
        MAX_PREWRITE_TIME: 0
          AVG_COMMIT_TIME: 0
          MAX_COMMIT_TIME: 0
   AVG_GET_COMMIT_TS_TIME: 0
   MAX_GET_COMMIT_TS_TIME: 0
  AVG_COMMIT_BACKOFF_TIME: 0
  MAX_COMMIT_BACKOFF_TIME: 0
    AVG_RESOLVE_LOCK_TIME: 0
    MAX_RESOLVE_LOCK_TIME: 0
AVG_LOCAL_LATCH_WAIT_TIME: 0
MAX_LOCAL_LATCH_WAIT_TIME: 0
           AVG_WRITE_KEYS: 0
           MAX_WRITE_KEYS: 0
           AVG_WRITE_SIZE: 0
           MAX_WRITE_SIZE: 0
     AVG_PREWRITE_REGIONS: 0
     MAX_PREWRITE_REGIONS: 0
            AVG_TXN_RETRY: 0
            MAX_TXN_RETRY: 0
        SUM_BACKOFF_TIMES: 0
            BACKOFF_TYPES: NULL
                  AVG_MEM: 16672
                  MAX_MEM: 16672
        AVG_AFFECTED_ROWS: 0
               FIRST_SEEN: 2019-12-22 16:18:25
                LAST_SEEN: 2019-12-22 16:19:13
        QUERY_SAMPLE_TEXT: select table_id, is_index, hist_id, distinct_count, version, null_count, tot_col_size, stats_ver, flag, correlation, last_analyze_pos from mysql.stats_histograms where table_id = 43
         PREV_SAMPLE_TEXT:
              PLAN_DIGEST: db05f9e134185031e52539c303d1930de3cf6e5d4881aee7ef97ba87a9a0a6d9
                     PLAN: 	Projection_4    	root	10	mysql.stats_histograms.table_id, mysql.stats_histograms.is_index, mysql.stats_histograms.hist_id, mysql.stats_histograms.distinct_count, mysql.stats_histograms.version, mysql.stats_histograms.null_count, mysql.stats_histograms.tot_col_size, mysql.stats_histograms.stats_ver, mysql.stats_histograms.flag, mysql.stats_histograms.correlation, mysql.stats_histograms.last_analyze_pos
	└─IndexLookUp_10	root	10
	  ├─IndexScan_8 	cop 	10	table:stats_histograms, index:table_id, is_index, hist_id, range:[43,43], keep order:false, stats:pseudo
	  └─TableScan_9 	cop 	10	table:stats_histograms, keep order:false, stats:pseudo

Code changes

  • N/A

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release note

  • And plan and plan_digest to the statement summary tables.
  • Change default value of configuration max-stmt-count in [stmt-summary] to 200.

@sre-bot
Copy link
Contributor Author

sre-bot commented Dec 30, 2019

/run-all-tests

@djshow832
Copy link
Contributor

/run-all-tests

Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@lonng lonng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bb7133 bb7133 merged commit e1d8b41 into pingcap:release-3.0 Jan 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants