Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add perf_schema quantile columns to collector #897

Merged
merged 2 commits into from
Nov 28, 2024

Conversation

matthewnolf
Copy link
Contributor

@matthewnolf matthewnolf commented Nov 25, 2024

This change appends the 3 quantile columns exposed in the events_statements_summary_by_digest table to the
perf_schema_events_statements collector so that they are available as a summary metric.

We believe this to be a useful addition so that the various quantiles (p95, p99, p99.5) are available to query and spot changes in performance for a given query. Our use case is to visualise these quantiles for a specific digest to provide more insight than just average query times.

The new summary metric mysql_perf_schema_events_statements_latency has 3 quantiles: 95, 99, 99.5 and in addition has a sum and count metric that matches the values of events_statements_seconds_total and events_statements_total respectively. These metrics may be considered redundant with this addition.

There is mention of adding quantiles in a previous change from a colleague here which wasn't further investigated.

Sample /metrics output that includes the new metrics:

# HELP mysql_perf_schema_events_statements_latency A summary of statement latency by digest
# TYPE mysql_perf_schema_events_statements_latency summary
mysql_perf_schema_events_statements_latency{digest="12fbbd91beb73676bfc7c0cf3cc73d8c29ccc0349a54e367556b6fb7d244a4f8",digest_text="SELECT DISTINCTROW `stack_id` FROM `jobs`",schema="hosted_exporters",quantile="95"} 0.000208929613
mysql_perf_schema_events_statements_latency{digest="12fbbd91beb73676bfc7c0cf3cc73d8c29ccc0349a54e367556b6fb7d244a4f8",digest_text="SELECT DISTINCTROW `stack_id` FROM `jobs`",schema="hosted_exporters",quantile="99"} 0.000239883291
mysql_perf_schema_events_statements_latency{digest="12fbbd91beb73676bfc7c0cf3cc73d8c29ccc0349a54e367556b6fb7d244a4f8",digest_text="SELECT DISTINCTROW `stack_id` FROM `jobs`",schema="hosted_exporters",quantile="999"} 0.001819700858
mysql_perf_schema_events_statements_latency_sum{digest="12fbbd91beb73676bfc7c0cf3cc73d8c29ccc0349a54e367556b6fb7d244a4f8",digest_text="SELECT DISTINCTROW `stack_id` FROM `jobs`",schema="hosted_exporters"} 2828.78780601
mysql_perf_schema_events_statements_latency_count{digest="12fbbd91beb73676bfc7c0cf3cc73d8c29ccc0349a54e367556b6fb7d244a4f8",digest_text="SELECT DISTINCTROW `stack_id` FROM `jobs`",schema="hosted_exporters"} 1.8455211e+07

@matthewnolf matthewnolf force-pushed the mn/add-quantile branch 2 times, most recently from e603762 to 10e4c0d Compare November 25, 2024 11:52
This change appends the 3 quantile columns exposed in the
events_statements_summary_by_digest table to the
perf_schema_events_statements collector so that they are available
as metrics

Signed-off-by: Matt Nolf <matt.nolf@grafana.com>
@cristiangreco
Copy link
Contributor

@SuperQ could use your thoughts here :)

This change replaces the individual quantile (95,99,999) metrics
with a summary that captures those respective quantiles. Enables
a more prometheus native representation of the data.

Signed-off-by: Matt Nolf <matt.nolf@grafana.com>
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, thanks!

@SuperQ SuperQ merged commit 2ef168b into prometheus:main Nov 28, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants