Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metrics for write procedure #831

Merged
merged 4 commits into from
Apr 12, 2023

Conversation

ShiKaiWi
Copy link
Member

@ShiKaiWi ShiKaiWi commented Apr 12, 2023

Which issue does this PR close?

Closes #

Rationale for this change

In order to improve the write performance, some metrics for the write procedure are necessary but missing now.

What changes are included in this PR?

  • Add metrics for all stages during the write procedure.

Are there any user-facing changes?

None.

How does this change test

Manual test.

This is what the metrics looks like:

table_write_duration_bucket{type="encode",le="0.01"} 32
table_write_duration_bucket{type="encode",le="0.02"} 43
table_write_duration_bucket{type="encode",le="0.04"} 45
table_write_duration_bucket{type="encode",le="0.08"} 45
table_write_duration_bucket{type="encode",le="0.16"} 45
table_write_duration_bucket{type="encode",le="0.32"} 45
table_write_duration_bucket{type="encode",le="0.64"} 45
table_write_duration_bucket{type="encode",le="1.28"} 45
table_write_duration_bucket{type="encode",le="2.56"} 45
table_write_duration_bucket{type="encode",le="5.12"} 45
table_write_duration_bucket{type="encode",le="10.24"} 45
table_write_duration_bucket{type="encode",le="20.48"} 45
table_write_duration_bucket{type="encode",le="40.96"} 45
table_write_duration_bucket{type="encode",le="+Inf"} 45
table_write_duration_sum{type="encode"} 0.454472457
table_write_duration_count{type="encode"} 45
table_write_duration_bucket{type="flush_wait",le="0.01"} 4
table_write_duration_bucket{type="flush_wait",le="0.02"} 4
table_write_duration_bucket{type="flush_wait",le="0.04"} 4
table_write_duration_bucket{type="flush_wait",le="0.08"} 4
table_write_duration_bucket{type="flush_wait",le="0.16"} 4
table_write_duration_bucket{type="flush_wait",le="0.32"} 4
table_write_duration_bucket{type="flush_wait",le="0.64"} 5
table_write_duration_bucket{type="flush_wait",le="1.28"} 5
table_write_duration_bucket{type="flush_wait",le="2.56"} 5
table_write_duration_bucket{type="flush_wait",le="5.12"} 5
table_write_duration_bucket{type="flush_wait",le="10.24"} 5
table_write_duration_bucket{type="flush_wait",le="20.48"} 5
table_write_duration_bucket{type="flush_wait",le="40.96"} 5
table_write_duration_bucket{type="flush_wait",le="+Inf"} 5
table_write_duration_sum{type="flush_wait"} 0.374436524
table_write_duration_count{type="flush_wait"} 5
table_write_duration_bucket{type="instance_flush_wait",le="0.01"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.02"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.04"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.08"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.16"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.32"} 0
table_write_duration_bucket{type="instance_flush_wait",le="0.64"} 0
table_write_duration_bucket{type="instance_flush_wait",le="1.28"} 0
table_write_duration_bucket{type="instance_flush_wait",le="2.56"} 0
table_write_duration_bucket{type="instance_flush_wait",le="5.12"} 0
table_write_duration_bucket{type="instance_flush_wait",le="10.24"} 0
table_write_duration_bucket{type="instance_flush_wait",le="20.48"} 0
table_write_duration_bucket{type="instance_flush_wait",le="40.96"} 0
table_write_duration_bucket{type="instance_flush_wait",le="+Inf"} 0
table_write_duration_sum{type="instance_flush_wait"} 0
table_write_duration_count{type="instance_flush_wait"} 0
table_write_duration_bucket{type="memtable",le="0.01"} 35
table_write_duration_bucket{type="memtable",le="0.02"} 39
table_write_duration_bucket{type="memtable",le="0.04"} 43
table_write_duration_bucket{type="memtable",le="0.08"} 45
table_write_duration_bucket{type="memtable",le="0.16"} 45
table_write_duration_bucket{type="memtable",le="0.32"} 45
table_write_duration_bucket{type="memtable",le="0.64"} 45
table_write_duration_bucket{type="memtable",le="1.28"} 45
table_write_duration_bucket{type="memtable",le="2.56"} 45
table_write_duration_bucket{type="memtable",le="5.12"} 45
table_write_duration_bucket{type="memtable",le="10.24"} 45
table_write_duration_bucket{type="memtable",le="20.48"} 45
table_write_duration_bucket{type="memtable",le="40.96"} 45
table_write_duration_bucket{type="memtable",le="+Inf"} 45
table_write_duration_sum{type="memtable"} 0.5338555669999999
table_write_duration_count{type="memtable"} 45
table_write_duration_bucket{type="preprocess",le="0.01"} 44
table_write_duration_bucket{type="preprocess",le="0.02"} 44
table_write_duration_bucket{type="preprocess",le="0.04"} 44
table_write_duration_bucket{type="preprocess",le="0.08"} 44
table_write_duration_bucket{type="preprocess",le="0.16"} 44
table_write_duration_bucket{type="preprocess",le="0.32"} 44
table_write_duration_bucket{type="preprocess",le="0.64"} 45
table_write_duration_bucket{type="preprocess",le="1.28"} 45
table_write_duration_bucket{type="preprocess",le="2.56"} 45
table_write_duration_bucket{type="preprocess",le="5.12"} 45
table_write_duration_bucket{type="preprocess",le="10.24"} 45
table_write_duration_bucket{type="preprocess",le="20.48"} 45
table_write_duration_bucket{type="preprocess",le="40.96"} 45
table_write_duration_bucket{type="preprocess",le="+Inf"} 45
table_write_duration_sum{type="preprocess"} 0.38392951199999986
table_write_duration_count{type="preprocess"} 45
table_write_duration_bucket{type="space_flush_wait",le="0.01"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.02"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.04"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.08"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.16"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.32"} 0
table_write_duration_bucket{type="space_flush_wait",le="0.64"} 0
table_write_duration_bucket{type="space_flush_wait",le="1.28"} 0
table_write_duration_bucket{type="space_flush_wait",le="2.56"} 0
table_write_duration_bucket{type="space_flush_wait",le="5.12"} 0
table_write_duration_bucket{type="space_flush_wait",le="10.24"} 0
table_write_duration_bucket{type="space_flush_wait",le="20.48"} 0
table_write_duration_bucket{type="space_flush_wait",le="40.96"} 0
table_write_duration_bucket{type="space_flush_wait",le="+Inf"} 0
table_write_duration_sum{type="space_flush_wait"} 0
table_write_duration_count{type="space_flush_wait"} 0
table_write_duration_bucket{type="stall",le="0.01"} 0
table_write_duration_bucket{type="stall",le="0.02"} 0
table_write_duration_bucket{type="stall",le="0.04"} 0
table_write_duration_bucket{type="stall",le="0.08"} 0
table_write_duration_bucket{type="stall",le="0.16"} 0
table_write_duration_bucket{type="stall",le="0.32"} 0
table_write_duration_bucket{type="stall",le="0.64"} 1
table_write_duration_bucket{type="stall",le="1.28"} 1
table_write_duration_bucket{type="stall",le="2.56"} 1
table_write_duration_bucket{type="stall",le="5.12"} 1
table_write_duration_bucket{type="stall",le="10.24"} 1
table_write_duration_bucket{type="stall",le="20.48"} 1
table_write_duration_bucket{type="stall",le="40.96"} 1
table_write_duration_bucket{type="stall",le="+Inf"} 1
table_write_duration_sum{type="stall"} 0.374154946
table_write_duration_count{type="stall"} 1
table_write_duration_bucket{type="wal",le="0.01"} 32
table_write_duration_bucket{type="wal",le="0.02"} 44
table_write_duration_bucket{type="wal",le="0.04"} 45
table_write_duration_bucket{type="wal",le="0.08"} 45
table_write_duration_bucket{type="wal",le="0.16"} 45
table_write_duration_bucket{type="wal",le="0.32"} 45
table_write_duration_bucket{type="wal",le="0.64"} 45
table_write_duration_bucket{type="wal",le="1.28"} 45
table_write_duration_bucket{type="wal",le="2.56"} 45
table_write_duration_bucket{type="wal",le="5.12"} 45
table_write_duration_bucket{type="wal",le="10.24"} 45
table_write_duration_bucket{type="wal",le="20.48"} 45
table_write_duration_bucket{type="wal",le="40.96"} 45
table_write_duration_bucket{type="wal",le="+Inf"} 45
table_write_duration_sum{type="wal"} 0.37082901800000007
table_write_duration_count{type="wal"} 45

Copy link
Contributor

@jiacai2050 jiacai2050 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShiKaiWi ShiKaiWi added this pull request to the merge queue Apr 12, 2023
Merged via the queue into apache:main with commit ad55f09 Apr 12, 2023
chunshao90 pushed a commit to chunshao90/ceresdb that referenced this pull request May 15, 2023
* feat: metrics for all write stages

* feat: record the metrics in the write procedure

* feat: rename some labels

* feat: add more metrics
@ShiKaiWi ShiKaiWi deleted the feat-write-metrics branch May 29, 2023 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants