configable budget for runtime

[gftea]

addressing #5696
First experiment with configurable coop budget
Also create an example project for benchmarking

[gftea]

@Darksonn @Noah-Kennedy @carllerche , I can't figure out why the minrust job is failing
https://github.com/tokio-rs/tokio/actions/runs/5205172992/jobs/9390335891

[gftea]

Hi,

I would like to breakdown the work into below plan

1. this PR will add only configuration api on runtime builder
2. will issue another PR for per task budget
3. will create seperate project for use cases implementation for benchmarking, either inside or outside tokio repo, please suggest your preference. @Darksonn @carllerche @Noah-Kennedy

[gftea]

Summary of benchmark

After running a similar duration, we can see with budget=32, it has better latency performance according to 99.9% quantile, stdev and mean value

high priority with budget = 32

[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 428775
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 321.4437572153228
[src/client.rs:72] hist.stdev() = 1093.6610468983138
[src/client.rs:72] hist.max() = 48671
[src/client.rs:72] hist.value_at_quantile(0.999) = 13919
[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 434070
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 321.55305595871664
[src/client.rs:72] hist.stdev() = 1094.422093579727
[src/client.rs:72] hist.max() = 48671
[src/client.rs:72] hist.value_at_quantile(0.999) = 13911
[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 439395
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 321.2118344541942
[src/client.rs:72] hist.stdev() = 1093.3720641192087
[src/client.rs:72] hist.max() = 48671
[src/client.rs:72] hist.value_at_quantile(0.999) = 13895

high priority with budget = 128

[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 426473
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 496.84516722043395
[src/client.rs:72] hist.stdev() = 1387.013451650282
[src/client.rs:72] hist.max() = 62847
[src/client.rs:72] hist.value_at_quantile(0.999) = 15767
[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 431781
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 494.767087945045
[src/client.rs:72] hist.stdev() = 1385.5939732739735
[src/client.rs:72] hist.max() = 62847
[src/client.rs:72] hist.value_at_quantile(0.999) = 15767
[src/client.rs:72] tag = "high priority client"
[src/client.rs:72] hist.len() = 437103
[src/client.rs:72] hist.min() = 0
[src/client.rs:72] hist.mean() = 492.5462785659203
[src/client.rs:72] hist.stdev() = 1383.6372255004694
[src/client.rs:72] hist.max() = 62847
[src/client.rs:72] hist.value_at_quantile(0.999) = 15767

machine

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              2
Core(s) per socket:              2
Socket(s):                       1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           140
Model name:                      11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
Stepping:                        1
CPU MHz:                         2995.200
BogoMIPS:                        5990.40
Hypervisor vendor:               Microsoft
Virtualization type:             full
L1d cache:                       96 KiB
L1i cache:                       64 KiB
L2 cache:                        2.5 MiB
L3 cache:                        12 MiB

[Darksonn]

This introduces an extra thread-local access on every poll. @carllerche Thoughts? I think you said that you were working on reducing them.

[carllerche]

We should be removing TLS calls in the hot path at this point. It should be possible to build this without any more TLS lookups.

I'm in the middle of a big scheduler refactor. It will be easier to make this change after the refactor lands. Ideally in 2-3 weeks.

[gftea]

For stable api, get_init_budget() return value directly instead of accessing the TLS, see below

pub(super) fn get_init_budget() -> Result<coop::Budget, AccessError> {
    #[cfg(not(tokio_unstable))]
    {
        Ok(coop::Budget::default())
    }

    #[cfg(tokio_unstable)]
    {
        CONTEXT.try_with(|ctx| ctx.initial_budget.get())
    }
}

[Darksonn]

I'm going to close this, because we don't want to add additional uses of the thread-local at this time.

[gftea]

I'm going to close this, because we don't want to add additional uses of the thread-local at this time.

@Darksonn , may I know what is plan forward, would we consider to work on this after some factors are done?

[Darksonn]

If you want to start working on this again with a different approach, then I'm open to that. You no longer need to wait for Carl's work to finish.