metrics: Improve flexibility of H2Histogram Configuration #6963

rcoh · 2024-11-08T15:12:12Z

Motivation

Based on feedback #6960, customers want more flexibility in histogram configuration.

The main change here is allowing precision for an H2 Histgoram to go all the way down to zero—there is not fundamental reason this wasn't possible before, but typically H2 histograms stop at P=2.

Solution

I've also added another optimization that truncates the number of buckets based on the requested max_value. This can save space by truncating in the middle of a bucket group.

Along the way and in the process of adding more tests, I found a couple of bugs which are fixed here:

The max_value in LogHistogramBuilder wasn't precisely respected
The max_value computed by LogHistogram was actually incorrect for some n/p combinations. The new version precisely considers bucket layout instead.

Darksonn · 2024-11-08T16:44:37Z

tokio/tests/rt_unstable_metrics.rs

+    for b in 0..num_buckets {
+        let range = metrics.poll_time_histogram_bucket_range(b);
+        let size = range.end - range.start;
+        println!("bucket {b}: {range:?} (size: {size:?})");


Can we assert their values here instead of printing?

Darksonn · 2024-11-08T16:45:54Z

tokio/src/runtime/metrics/histogram/h2_histogram.rs

-            panic!("precision must be >= 2");
-        };
-        if p > 10 {
+        if p > MAX_PRECISION {
            panic!("precision must be <= 10");


Suggested change

panic!("precision must be <= 10");

panic!("precision must be <= {}", MAX_PRECISION);

Darksonn · 2024-11-08T16:47:07Z

tokio/src/runtime/metrics/histogram/h2_histogram.rs

+    /// To match the behavior of the previous log histogram implementation, use
+    /// `builder.precision_exact(0)`. This creates a histogram where each bucket is twice the size
+    /// of the previous value.


I'd revert the order of this sentence so you first say that passing 0 results in buckets with doubling being used for the size, and then you can comment that this matches the behavior used in the legacy implementation.

Darksonn · 2024-11-08T16:47:59Z

tokio/src/runtime/builder.rs

@@ -1245,8 +1245,30 @@ impl Builder {
        ///     .unwrap();
        /// ```
        ///
+        /// Configure a `LogHistogram` with similar behavior to [`HistogramScale::Log`]


This sentence is confusing. I'm guessing HistogramScale::Log is the old implementation?

yes clarified

Darksonn · 2024-11-08T16:48:05Z

tokio/src/runtime/builder.rs

+        /// ```rust
+        ///


Suggested change

/// ```rust

///

/// ```rust

tokio/src/runtime/metrics/histogram/h2_histogram.rs

Darksonn · 2024-11-21T16:29:25Z

Any status update on this?

rcoh · 2024-11-21T16:45:48Z

Sorry I went on parental leave, I will have someone from my team follow up

…

On Thu, Nov 21, 2024, 11:29 AM Alice Ryhl ***@***.***> wrote: Any status update on this? — Reply to this email directly, view it on GitHub <#6963 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADYKZ73MH7JJPJLKHMEDML2BYC7VAVCNFSM6AAAAABRNWIB2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJRG4YDSMZRGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

hds · 2024-11-21T18:21:04Z

@rcoh:

Sorry I went on parental leave

Congratulations!!

rcoh · 2024-12-15T11:33:23Z

This is ready btw @Darksonn

Darksonn · 2024-12-15T12:50:06Z

@hds or @Noah-Kennedy can you review?

Noah-Kennedy · 2024-12-15T16:35:23Z

I'll review today

Noah-Kennedy · 2024-12-16T13:23:25Z

I'd like @hds to take a look actually here, as I have not been involved with the histogram too much.

hds

This is good from my side, but there are still a couple of comments from @Darksonn which I think haven't been addressed.

Based on feedback tokio-rs#6960, customers want more flexibility in histogram configuration. The main change here is allowing precision for an H2 Histgoram to go all the way down to zero—there is not fundamental reason this wasn't possible before, but typically H2 histograms stop at P=2. I've also added another optimization that truncates the number of buckets based on the requested max_value. This can save space by truncating in the middle of a bucket group. Along the way and in the process of adding more tests, I found a couple of bugs which are fixed here: 1. The `max_value` in LogHistogramBuilder wasn't precisely respected 2. The `max_value` computed by `LogHistogram` was actually incorrect for some n/p combinations. The new version precisely considers bucket layout instead.

Darksonn · 2025-01-04T12:17:37Z

I'll review this once I'm back from vacation next week. :)

Darksonn

Thanks.

⚠️ Dependabot is rebasing this PR ⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. Bumps tokio from 1.42.0 to 1.43.0. Release notes Sourced from tokio's releases. Tokio v1.43.0 1.43.0 (Jan 8th, 2025) Added net: add UdpSocket::peek methods (#7068) net: add support for Haiku OS (#7042) process: add Command::into_std() (#7014) signal: add SignalKind::info on illumos (#6995) signal: add support for realtime signals on illumos (#7029) Fixed io: don't call set_len before initializing vector in Blocking (#7054) macros: suppress clippy::needless_return in #[tokio::main] (#6874) runtime: fix thread parking on WebAssembly (#7041) Changes chore: use unsync loads for unsync_load (#7073) io: use Buf::put_bytes in Repeat read impl (#7055) task: drop the join waker of a task eagerly (#6986) Changes to unstable APIs metrics: improve flexibility of H2Histogram Configuration (#6963) taskdump: add accessor methods for backtrace (#6975) Documented io: clarify ReadBuf::uninit allows initialized buffers as well (#7053) net: fix ambiguity in TcpStream::try_write_vectored docs (#7067) runtime: fix LocalRuntime doc links (#7074) sync: extend documentation for watch::Receiver::wait_for (#7038) sync: fix typos in OnceCell docs (#7047) #6874: tokio-rs/tokio#6874 #6963: tokio-rs/tokio#6963 #6975: tokio-rs/tokio#6975 #6986: tokio-rs/tokio#6986 #6995: tokio-rs/tokio#6995 #7014: tokio-rs/tokio#7014 #7029: tokio-rs/tokio#7029 #7038: tokio-rs/tokio#7038 #7041: tokio-rs/tokio#7041 #7042: tokio-rs/tokio#7042 #7047: tokio-rs/tokio#7047 #7053: tokio-rs/tokio#7053 #7054: tokio-rs/tokio#7054 #7055: tokio-rs/tokio#7055 ... (truncated) Commits 5f3296d chore: prepare Tokio v1.43.0 (#7079) cc974a6 chore: prepare tokio-macros v2.5.0 (#7078) 15495fd metrics: improve flexibility of H2Histogram Configuration (#6963) ad41834 io: don't call set_len before initializing vector in Blocking (#7054) bd3e857 runtime: move is_join_waker_set assertion in unset_waker (#7072) 15f7366 runtime: fix LocalRuntime doc links (#7074) fd2048d ci: split miri jobs into unit and integration tests (#7071) e8f3915 chore: use unsync loads for unsync_load (#7073) 67f1277 net: fix ambiguity in TcpStream::try_write_vectored docs (#7067) 463502c io: clarify ReadBuf::uninit allows initialized buffers as well (#7053) Additional commits viewable in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase. Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: @dependabot rebase will rebase this PR @dependabot recreate will recreate this PR, overwriting any edits that have been made to it @dependabot merge will merge this PR after your CI passes on it @dependabot squash and merge will squash and merge this PR after your CI passes on it @dependabot cancel merge will cancel a previously requested merge and block automerging @dependabot reopen will reopen this PR if it is closed @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Darksonn added A-tokio Area: The main tokio crate M-metrics Module: tokio/runtime/metrics labels Nov 8, 2024

Darksonn reviewed Nov 8, 2024

View reviewed changes

hds reviewed Nov 8, 2024

View reviewed changes

tokio/src/runtime/metrics/histogram/h2_histogram.rs Outdated Show resolved Hide resolved

tokio/src/runtime/metrics/histogram/h2_histogram.rs Outdated Show resolved Hide resolved

rcoh mentioned this pull request Nov 9, 2024

The H2 log algorithm for the poll time histogram is too granular for prometheus export #6960

Closed

rcoh force-pushed the allow-p-0 branch 2 times, most recently from 009d531 to eaffd4b Compare November 25, 2024 15:25

hds approved these changes Dec 18, 2024

View reviewed changes

rcoh and others added 3 commits December 26, 2024 15:25

Misc cleanups of docs & tests

ddbdbc3

Clarify doc comment

5f8172c

rcoh force-pushed the allow-p-0 branch from d9be942 to 5f8172c Compare December 26, 2024 20:25

rcoh requested a review from Darksonn January 3, 2025 20:15

Darksonn approved these changes Jan 8, 2025

View reviewed changes

Darksonn merged commit 15495fd into tokio-rs:master Jan 8, 2025
81 checks passed

Darksonn mentioned this pull request Jan 8, 2025

chore: prepare Tokio v1.43.0 #7079

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

metrics: Improve flexibility of H2Histogram Configuration #6963

metrics: Improve flexibility of H2Histogram Configuration #6963

rcoh commented Nov 8, 2024

Darksonn Nov 8, 2024

Darksonn Nov 8, 2024

Darksonn Nov 8, 2024

Darksonn Nov 8, 2024

rcoh Dec 26, 2024

Darksonn Nov 8, 2024

Darksonn commented Nov 21, 2024

rcoh commented Nov 21, 2024 via email

hds commented Nov 21, 2024

rcoh commented Dec 15, 2024

Darksonn commented Dec 15, 2024

Noah-Kennedy commented Dec 15, 2024

Noah-Kennedy commented Dec 16, 2024

hds left a comment

Darksonn commented Jan 4, 2025

Darksonn left a comment

	panic!("precision must be <= 10");
	panic!("precision must be <= {}", MAX_PRECISION);

metrics: Improve flexibility of H2Histogram Configuration #6963

metrics: Improve flexibility of H2Histogram Configuration #6963

Conversation

rcoh commented Nov 8, 2024

Motivation

Solution

Darksonn Nov 8, 2024

Choose a reason for hiding this comment

Darksonn Nov 8, 2024

Choose a reason for hiding this comment

Darksonn Nov 8, 2024

Choose a reason for hiding this comment

Darksonn Nov 8, 2024

Choose a reason for hiding this comment

rcoh Dec 26, 2024

Choose a reason for hiding this comment

Darksonn Nov 8, 2024

Choose a reason for hiding this comment

Darksonn commented Nov 21, 2024

rcoh commented Nov 21, 2024 via email

hds commented Nov 21, 2024

rcoh commented Dec 15, 2024

Darksonn commented Dec 15, 2024

Noah-Kennedy commented Dec 15, 2024

Noah-Kennedy commented Dec 16, 2024

hds left a comment

Choose a reason for hiding this comment

Darksonn commented Jan 4, 2025

Darksonn left a comment

Choose a reason for hiding this comment