Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statistics: skip update global count and modify_count if only the partition is locked #47259

Merged
merged 8 commits into from
Sep 26, 2023

Conversation

Rustin170506
Copy link
Member

@Rustin170506 Rustin170506 commented Sep 25, 2023

What problem does this PR solve?

Issue Number: ref #46351

Problem Summary:

What is changed and how it works?

skip update global count and modify_count if the table is locked.

Check List

Tests

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 25, 2023
@tiprow
Copy link

tiprow bot commented Sep 25, 2023

Hi @hi-rustin. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 25, 2023
@codecov
Copy link

codecov bot commented Sep 25, 2023

Codecov Report

Merging #47259 (f376a8a) into master (df76319) will decrease coverage by 0.3268%.
Report is 24 commits behind head on master.
The diff coverage is 63.1578%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #47259        +/-   ##
================================================
- Coverage   72.9671%   72.6404%   -0.3268%     
================================================
  Files          1340       1367        +27     
  Lines        400269     408804      +8535     
================================================
+ Hits         292065     296957      +4892     
- Misses        89270      93009      +3739     
+ Partials      18934      18838        -96     
Flag Coverage Δ
integration 33.7586% <0.0000%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9913% <ø> (ø)
parser 84.9340% <ø> (-0.0036%) ⬇️
br 48.7348% <ø> (-4.3459%) ⬇️

@Rustin170506
Copy link
Member Author

Rustin170506 commented Sep 25, 2023

Test steps:

  1. Create table: create table pt(a int, b varchar(10), index idx_b (b)) partition by range(a) (partition p0 values less than (10), partition p1 values less than (20), partition p2 values less than (30));
  2. Insert data: insert into pt(a, b) values(1,'a'); insert into pt(a, b) values(15,'a');
  3. Wait for analysis and check the result: select * from mysql.stats_meta;
mysql> select * from mysql.stats_meta;
+--------------------+----------+--------------+-------+----------+
| version            | table_id | modify_count | count | snapshot |
+--------------------+----------+--------------+-------+----------+
| 444517182499979273 |      103 |            1 |     1 |        0 |
| 444517182499979266 |      104 |            1 |     1 |        0 |
| 444517175068196868 |      105 |            0 |     0 |        0 |
| 444517182499979273 |      102 |            2 |     2 |        0 |
+--------------------+----------+--------------+-------+----------+
4 rows in set (0.01 sec)
  1. Only lock one partition: lock stats pt partition p1;
mysql> lock stats pt partition p1;
Query OK, 0 rows affected (0.01 sec)
mysql> show stats_locked;
+---------+------------+----------------+--------+
| Db_name | Table_name | Partition_name | Status |
+---------+------------+----------------+--------+
| test    | pt         | p1             | locked |
+---------+------------+----------------+--------+
1 row in set (0.00 sec)
  1. insert some data: insert into pt(a, b) values(15,'a');insert into pt(a, b) values(8,'a');
  2. Wait for analysis and check the result: select * from mysql.stats_meta;
mysql> select * from mysql.stats_meta;
+--------------------+----------+--------------+-------+----------+
| version            | table_id | modify_count | count | snapshot |
+--------------------+----------+--------------+-------+----------+
| 444517229685899274 |      103 |            2 |     2 |        0 |
| 444517182499979266 |      104 |            1 |     1 |        0 |
| 444517175068196868 |      105 |            0 |     0 |        0 |
| 444517229685899274 |      102 |            3 |     3 |        0 |
+--------------------+----------+--------------+-------+----------+
4 rows in set (0.00 sec
  1. Unlock and check the result again: unlock stats pt partition p1; select * from mysql.stats_meta;
mysql> unlock stats pt partition p1;
Query OK, 0 rows affected (0.02 sec)

mysql> select * from mysql.stats_meta;
+--------------------+----------+--------------+-------+----------+
| version            | table_id | modify_count | count | snapshot |
+--------------------+----------+--------------+-------+----------+
| 444517229685899274 |      103 |            2 |     2 |        0 |
| 444517229685899265 |      104 |            2 |     2 |        0 |
| 444517175068196868 |      105 |            0 |     0 |        0 |
| 444517229685899265 |      102 |            4 |     4 |        0 |
+--------------------+----------+--------------+-------+----------+
4 rows in set (0.01 sec)
  1. Wait for another round of analysis and check the result: select * from mysql.stats_meta; and nothing changes.
mysql> select * from mysql.stats_meta;
+--------------------+----------+--------------+-------+----------+
| version            | table_id | modify_count | count | snapshot |
+--------------------+----------+--------------+-------+----------+
| 444517229685899274 |      103 |            2 |     2 |        0 |
| 444517229685899265 |      104 |            2 |     2 |        0 |
| 444517175068196868 |      105 |            0 |     0 |        0 |
| 444517229685899265 |      102 |            4 |     4 |        0 |
+--------------------+----------+--------------+-------+----------+
4 rows in set (0.00 sec)

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Sep 25, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 25, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-09-25 12:31:58.503063495 +0000 UTC m=+278108.221405710: ☑️ agreed by winoros.
  • 2023-09-25 12:45:49.790298773 +0000 UTC m=+278939.508640990: ☑️ agreed by qw4990.

@Rustin170506 Rustin170506 changed the title statistics: skip update global count and modify_count if table is locked statistics: skip update global count and modify_count if partition is locked Sep 25, 2023
@time-and-fate
Copy link
Member

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 25, 2023
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 26, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Sep 26, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: qw4990, time-and-fate, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [qw4990,time-and-fate,winoros]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@time-and-fate
Copy link
Member

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 26, 2023
@Rustin170506
Copy link
Member Author

/unhold

@Rustin170506 Rustin170506 changed the title statistics: skip update global count and modify_count if partition is locked statistics: skip update global count and modify_count if only the partition is locked Sep 26, 2023
@ti-chi-bot ti-chi-bot bot merged commit c25fd6f into pingcap:master Sep 26, 2023
@Rustin170506 Rustin170506 deleted the rustin-patch-fix-stats branch September 26, 2023 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants