stmtsummary: add request-units info in statements_summary #49504

glorv · 2023-12-15T08:53:48Z

What problem does this PR solve?

Issue Number: ref #49318

Problem Summary:

What changed and how does it work?

Add following 7 columns in information_schema.statements_summary:

  `MAX_REQUEST_UNIT_READ` double unsigned NOT NULL COMMENT 'Max read request-unit cost of these statements',
  `AVG_REQUEST_UNIT_READ` double unsigned NOT NULL COMMENT 'Average read request-unit cost of these statements',
  `MAX_REQUEST_UNIT_WRITE` double unsigned NOT NULL COMMENT 'Max write request-unit cost of these statements',
  `AVG_REQUEST_UNIT_WRITE` double unsigned NOT NULL COMMENT 'Average write request-unit cost of these statements',
  `MAX_QUEUED_RC_TIME` bigint(22) unsigned NOT NULL COMMENT 'Max time of waiting for available request-units',
  `AVG_QUEUED_RC_TIME` bigint(22) unsigned NOT NULL COMMENT 'Max time of waiting for available request-units',
  `RESOURCE_GROUP` varchar(64) DEFAULT NULL COMMENT 'Bind resource group name'

And add resource_group_name in statements_summary's aggration key, so after this change, the same sql executed by under different resource groups with be collected as different record.

Result example:

mysql> select DIGEST_TEXT, MAX_REQUEST_UNIT_READ, AVG_REQUEST_UNIT_READ, MAX_REQUEST_UNIT_WRITE, AVG_REQUEST_UNIT_WRITE, MAX_QUEUED_RC_TIME, AVG_QUEUED_RC_TIME, RESOURCE_GROUP from statements_summary where AVG_REQUEST_UNIT_READ > 0 limit 1\G;
*************************** 1. row ***************************
           DIGEST_TEXT: select `c` from `sbtest1` where `id` = ?
 MAX_REQUEST_UNIT_READ: 0.7367986354166667
 AVG_REQUEST_UNIT_READ: 0.5131357532322988
MAX_REQUEST_UNIT_WRITE: 0
AVG_REQUEST_UNIT_WRITE: 0
    MAX_QUEUED_RC_TIME: 212279751
    AVG_QUEUED_RC_TIME: 35929510
        RESOURCE_GROUP: rg1
1 row in set (0.02 sec)

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2023-12-15T08:54:03Z

Hi @glorv. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

glorv · 2023-12-15T08:55:16Z

/cc @songrijie

codecov · 2023-12-15T09:14:26Z

Codecov Report

Merging #49504 (95e5f9a) into master (87f1a4a) will increase coverage by 0.4377%.
Report is 4 commits behind head on master.
The diff coverage is 59.2857%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #49504        +/-   ##
================================================
+ Coverage   70.9585%   71.3962%   +0.4377%     
================================================
  Files          1368       1428        +60     
  Lines        397088     415974     +18886     
================================================
+ Hits         281768     296990     +15222     
- Misses        95622     100188      +4566     
+ Partials      19698      18796       -902

Flag	Coverage Δ
integration	`43.8792% <59.2857%> (?)`
unit	`70.9560% <ø> (-0.0026%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`53.9663% <ø> (ø)`
parser	`∅ <ø> (∅)`
br	`47.2371% <ø> (-5.6680%)`	⬇️

glorv · 2023-12-15T09:18:15Z

/retest

tiprow · 2023-12-15T09:18:38Z

@glorv: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

glorv · 2023-12-15T09:18:55Z

/ok-to-test

…ents-summary

glorv · 2023-12-20T03:22:27Z

/retest

…ents-summary

nolouch

lgtm

nolouch · 2023-12-21T07:25:45Z

pkg/executor/adapter.go

@@ -1950,6 +1954,9 @@ func (a *ExecStmt) SummaryStmt(succ bool) {
 		Prepared:            a.isPreparedStmt,
 		KeyspaceName:        keyspaceName,
 		KeyspaceID:          keyspaceID,
+		RUDetail:            ruDetail,
+		// TODO: use sessVars.StmtCtx.ResourceGroupName


already use sessVars.StmtCtx.ResourceGroupName?

nolouch

lgtm

ti-chi-bot · 2023-12-21T08:59:39Z

@nolouch: Your lgtm message is repeated, so it is ignored.

In response to this:

lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

glorv · 2023-12-21T09:34:12Z

@tangenta PTAL, thanks

glorv · 2023-12-21T10:37:04Z

/retest

glorv · 2023-12-22T03:04:52Z

@mornyx @tangenta PTAL

CabinfeverB

rest LGTM

CabinfeverB · 2023-12-25T05:41:21Z

go.mod

@@ -103,7 +103,7 @@ require (
 	github.com/stretchr/testify v1.8.4
 	github.com/tdakkota/asciicheck v0.2.0
 	github.com/tiancaiamao/appdash v0.0.0-20181126055449-889f96f722a2
-	github.com/tikv/client-go/v2 v2.0.8-0.20231219052137-6f9ba8327b75
+	github.com/tikv/client-go/v2 v2.0.8-0.20231220023527-4ce1e45671b3


The client-go version is ahead of this. pls merge master.

ti-chi-bot · 2023-12-25T05:53:44Z

[LGTM Timeline notifier]

Timeline:

2023-12-21 08:59:33.547830929 +0000 UTC m=+1124264.585057860: ☑️ agreed by nolouch.
2023-12-25 05:53:42.952536275 +0000 UTC m=+1458713.989763202: ☑️ agreed by CabinfeverB.

…ents-summary

ti-chi-bot · 2023-12-25T06:16:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CabinfeverB, nolouch, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tangenta]
~~pkg/infoschema/OWNERS~~ [tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

glorv · 2023-12-25T06:24:20Z

/retest

ti-chi-bot · 2024-02-01T06:20:00Z

In response to a cherrypick label: new pull request created to branch release-7.5: #50879.

) ref pingcap#49318

ti-chi-bot · 2024-02-01T06:20:01Z

In response to a cherrypick label: failed to apply #49504 on top of branch "release-7.6":

failed to git commit: exit status 1

ti-chi-bot · 2024-02-01T06:21:14Z

In response to a cherrypick label: failed to apply #49504 on top of branch "release-7.6":

failed to git commit: exit status 1

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2024-02-01T06:21:16Z

In response to a cherrypick label: new pull request created to branch release-7.1: #50880.

ti-chi-bot · 2024-02-01T06:21:52Z

In response to a cherrypick label: failed to apply #49504 on top of branch "release-7.6":

failed to git commit: exit status 1

…50879) ref #49318

add request-units info in statments_summary

acd1118

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 15, 2023

glorv requested a review from nolouch December 15, 2023 08:54

ti-chi-bot bot requested a review from songrijie December 15, 2023 08:55

update bazel

b99b6ea

ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Dec 15, 2023

glorv added 6 commits December 15, 2023 17:36

add missing function comments

8555b5e

Merge branch 'master' of ssh://github.com/pingcap/tidb into rc-statem…

afcdbb4

…ents-summary

fix test

42edd8f

update client-go

52f2691

Merge branch 'master' of ssh://github.com/pingcap/tidb into rc-statem…

041e1d1

…ents-summary

update bazel

ad51e75

glorv added 2 commits December 21, 2023 11:38

Merge branch 'master' of ssh://github.com/pingcap/tidb into rc-statem…

c4b3ac9

…ents-summary

use statement resource-group-name

f30f49c

nolouch approved these changes Dec 21, 2023

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Dec 21, 2023

remove outdated comment

8bd9b5d

glorv requested a review from tangenta December 21, 2023 09:33

CabinfeverB approved these changes Dec 25, 2023

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Dec 25, 2023

Merge branch 'master' of ssh://github.com/pingcap/tidb into rc-statem…

95e5f9a

…ents-summary

tangenta approved these changes Dec 25, 2023

View reviewed changes

ti-chi-bot bot added the approved label Dec 25, 2023

ti-chi-bot bot merged commit 946bcfc into pingcap:master Dec 25, 2023
17 of 18 checks passed

nolouch added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Feb 1, 2024

ti-chi-bot added the needs-cherry-pick-release-7.6 label Feb 1, 2024

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Feb 1, 2024

stmtsummary: add request-units info in statements_summary (pingcap#49504

08bc185

) ref pingcap#49318

ti-chi-bot mentioned this pull request Feb 1, 2024

stmtsummary: add request-units info in statements_summary (#49504) #50879

Merged

13 tasks

nolouch added needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. and removed needs-cherry-pick-release-7.6 labels Feb 1, 2024

ti-chi-bot added needs-cherry-pick-release-7.6 labels Feb 1, 2024

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Feb 1, 2024

This is an automated cherry-pick of pingcap#49504

f35285b

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Feb 1, 2024

stmtsummary: add request-units info in statements_summary (#49504) #50880

Closed

13 tasks

ti-chi-bot removed the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Feb 18, 2024

ti-chi-bot bot pushed a commit that referenced this pull request Feb 19, 2024

stmtsummary: add request-units info in statements_summary (#49504) (#…

def88be

…50879) ref #49318

ti-chi-bot removed the needs-cherry-pick-release-7.1 Should cherry pick this PR to release-7.1 branch. label Apr 18, 2024

glorv deleted the rc-statements-summary branch September 24, 2024 08:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stmtsummary: add request-units info in statements_summary #49504

stmtsummary: add request-units info in statements_summary #49504

glorv commented Dec 15, 2023

tiprow bot commented Dec 15, 2023

glorv commented Dec 15, 2023

codecov bot commented Dec 15, 2023 •

edited

Loading

glorv commented Dec 15, 2023

tiprow bot commented Dec 15, 2023

glorv commented Dec 15, 2023

glorv commented Dec 20, 2023

nolouch left a comment

nolouch Dec 21, 2023

glorv Dec 21, 2023

nolouch left a comment

ti-chi-bot bot commented Dec 21, 2023

glorv commented Dec 21, 2023

glorv commented Dec 21, 2023

glorv commented Dec 22, 2023

CabinfeverB left a comment

CabinfeverB Dec 25, 2023

glorv Dec 25, 2023

ti-chi-bot bot commented Dec 25, 2023

ti-chi-bot bot commented Dec 25, 2023

glorv commented Dec 25, 2023

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

stmtsummary: add request-units info in statements_summary #49504

stmtsummary: add request-units info in statements_summary #49504

Conversation

glorv commented Dec 15, 2023

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

tiprow bot commented Dec 15, 2023

glorv commented Dec 15, 2023

codecov bot commented Dec 15, 2023 • edited Loading

Codecov Report

glorv commented Dec 15, 2023

tiprow bot commented Dec 15, 2023

glorv commented Dec 15, 2023

glorv commented Dec 20, 2023

nolouch left a comment

Choose a reason for hiding this comment

nolouch Dec 21, 2023

Choose a reason for hiding this comment

glorv Dec 21, 2023

Choose a reason for hiding this comment

nolouch left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Dec 21, 2023

glorv commented Dec 21, 2023

glorv commented Dec 21, 2023

glorv commented Dec 22, 2023

CabinfeverB left a comment

Choose a reason for hiding this comment

CabinfeverB Dec 25, 2023

Choose a reason for hiding this comment

glorv Dec 25, 2023

Choose a reason for hiding this comment

ti-chi-bot bot commented Dec 25, 2023

[LGTM Timeline notifier]

ti-chi-bot bot commented Dec 25, 2023

glorv commented Dec 25, 2023

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

ti-chi-bot commented Feb 1, 2024

codecov bot commented Dec 15, 2023 •

edited

Loading