Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metric: replace QueryDurationHistogram's "general" type to more detail stmt type #8819

Merged
merged 6 commits into from
Dec 29, 2018
Merged

metric: replace QueryDurationHistogram's "general" type to more detail stmt type #8819

merged 6 commits into from
Dec 29, 2018

Conversation

lysu
Copy link
Contributor

@lysu lysu commented Dec 25, 2018

What problem does this PR solve?

current Prometheus only display QueryDurationHistogram with internal and general, for general we often need more detail about which kind of SQL become slow.

What is changed and how it works?

replace QueryDurationHistogram's "general" type to more detail stmt type

why choose to replace general? add new dimensions will make time serial x2 per host, +n per host

corner case:

  1. SQL like insert into tbl select a, b from tbl2 will be recorded as root stmt type ---- Insert
  2. Multiple SQL sql1;sql2 will use last stmt's type

Check List

Tests

  • Exists Test Case

Code changes

  • Collection root stmt type in ProcessInfo
  • Record metric

Side effects

  • N/A

Related changes

  • Need to update the documentation
  • Need to be included in the release note

This change is Reviewable

@lysu
Copy link
Contributor Author

lysu commented Dec 25, 2018

/run-all-tests

executor/adapter.go Outdated Show resolved Hide resolved
executor/adapter.go Outdated Show resolved Hide resolved
server/conn.go Outdated
cc.ctx.SetProcessInfo("", t, mysql.ComSleep)
stmtType := cc.ctx.ShowProcess().StmtType
if stmtType != "" {
metrics.StmtDurationHistogram.WithLabelValues(stmtType).Observe(time.Since(t).Seconds())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we handle the metrics at the same place as QueryDurationHistogram?

@eurekaka
Copy link
Contributor

/run-unit-test

Copy link

@imtbkcat imtbkcat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: lysu <sulifx@gmail.com>
Signed-off-by: lysu <sulifx@gmail.com>
@imtbkcat imtbkcat added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 26, 2018
Signed-off-by: lysu <sulifx@gmail.com>
Signed-off-by: lysu <sulifx@gmail.com>
@lysu lysu changed the title metric: add stmt duration metric metric: replace QueryDurationHistogram's "general" type to more detail stmt type Dec 26, 2018
if stmtType != "" {
sqlType = stmtType
}
metrics.QueryDurationHistogram.WithLabelValues(sqlType).Observe(time.Since(startTime).Seconds())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do QueryDurationHistogram and QueryTotalCounter use the different label

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a "historical reasons", old code choose COMMAND as count label and none label for the duration,

To add duration label, IMO stmtType(like insert/select) is more useful than command(like COM_QUERY).

why don't change QueryTotalCounter is maybe sometime we need that, and we also have another StmtNodeCounter to cover "count by stmtType" although has some litte different.

so after this we can see:

  • count of command -> QueryTotalCounter
  • count of stmt -> StmtNodeCounter
  • duration of query -> QueryDurationHistogram and optional switch to stmtType view

:D

Copy link
Member

@jackysp jackysp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jackysp
Copy link
Member

jackysp commented Dec 28, 2018

/run-all-tests

1 similar comment
@lysu
Copy link
Contributor Author

lysu commented Dec 28, 2018

/run-all-tests

@lysu
Copy link
Contributor Author

lysu commented Dec 28, 2018

/rebuild

@lysu
Copy link
Contributor Author

lysu commented Dec 28, 2018

/run-all-tests

@lysu
Copy link
Contributor Author

lysu commented Dec 29, 2018

/run-unit-test

@lysu
Copy link
Contributor Author

lysu commented Dec 29, 2018

/run-unit-test

@zz-jason zz-jason merged commit 7a96642 into pingcap:master Dec 29, 2018
@zz-jason zz-jason added status/LGT2 Indicates that a PR has LGTM 2. status/all tests passed and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 29, 2018
yu34po pushed a commit to yu34po/tidb that referenced this pull request Jan 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/metrics status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants