metrics: add running metrics for grpc #639
Conversation
Signed-off-by: bufferflies <1045931706@qq.com>
Signed-off-by: bufferflies <1045931706@qq.com>
Signed-off-by: GitHub <noreply@github.com>
There was a problem hiding this comment.
It's better not to directly introduce Prometheus. There are two ways to do it:
- Use the monitoring infrastructure that comes with grpc (C++ counter/histogram), like this PR Support collecting gRPC core metrics #625.
- Add a feature to make Prometheus an optional dependency.
Signed-off-by: bufferflies <1045931706@qq.com>
Signed-off-by: bufferflies <1045931706@qq.com>
good idea, I had add new feature("promethes ") to check the dependency. |
|
/ping @overvenus |
Cargo.toml
Outdated
| @@ -27,6 +27,8 @@ prost = { version = "0.11", optional = true } | |||
| bytes = { version = "1.0", optional = true } | |||
| log = "0.4" | |||
| parking_lot = "0.12" | |||
| prometheus = { version = "0.13", default-features = false } | |||
There was a problem hiding this comment.
| prometheus = { version = "0.13", default-features = false } | |
| prometheus = { version = "0.13", default-features = false, optional = true } |
| @@ -42,8 +44,9 @@ members = [ | |||
| exclude = ["xtask"] | |||
|
|
|||
| [features] | |||
| default = ["protobuf-codec", "boringssl"] | |||
| default = ["protobuf-codec", "boringssl","prometheus"] | |||
There was a problem hiding this comment.
Maybe we should not enable the "prometheus" by default.
There was a problem hiding this comment.
It maybe bring the resouce usage.
Cargo.toml
Outdated
| _secure = [] | ||
| prometheus = [] |
There was a problem hiding this comment.
| prometheus = [] | |
| prometheus = ["dep:prometheus"] |
src/env.rs
Outdated
|
|
||
| #[cfg(feature = "prometheus")] | ||
| pub struct GRPCRunner { | ||
| cq_next_duration_his: Histogram, |
There was a problem hiding this comment.
Better use LocalHistogram and periodically flush(e.g. for every 1k calls) to reduce the overhead.
src/env.rs
Outdated
| cq_next_duration_his: Histogram, | ||
| execute_duration_his: Histogram, | ||
| wait_duration_his: Histogram, | ||
| event_counter: [GenericCounter<AtomicU64>; 6], |
There was a problem hiding this comment.
| event_counter: [GenericCounter<AtomicU64>; 6], | |
| event_counter: [Counter; 6], |
src/metrics.rs
Outdated
| "grpc_pool_execute_duration", | ||
| "Bucketed histogram of grpc pool execute duration for every time", | ||
| &["name"], | ||
| exponential_buckets(1e-7, 2.0, 20).unwrap() // 100ns ~ 100ms |
There was a problem hiding this comment.
100ms upper bound is too small here, maybe set the range to 10us~10s is better here.
bufferflies
force-pushed
the
metrics
branch
2 times, most recently
from
2f2cfd2
to
cf7cab6
Compare
Signed-off-by: tongjian <jian.tong@bytedance.com>
|
@overvenus PTAL |
close: #640
the metrics after this pr