Benchmark error: "Resource temporarily unavailable"

[zamazan4ik]

What happened:
I get an error during the benchmark run:

cargo bench --bench throughput
   Compiling quilkin v0.8.0-dev (/home/zamazan4ik/open_source/quilkin)
    Finished bench [optimized] target(s) in 6.11s
     Running benches/throughput.rs (target/release/deps/throughput-3dc8e04098c6256c)
Benchmarking readwrite/direct-read/254 bytes: Analyzingthread '<unnamed>' panicked at benches/throughput.rs:159:61:
called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
readwrite/direct-read/254 bytes
                        time:   [12.967 µs 12.977 µs 12.986 µs]
                        thrpt:  [18.653 MiB/s 18.667 MiB/s 18.681 MiB/s]
                 change:
                        time:   [+2.1734% +2.3202% +2.4654%] (p = 0.00 < 0.05)
                        thrpt:  [-2.4061% -2.2676% -2.1272%]
                        Performance has regressed.
Found 1093 outliers among 10000 measurements (10.93%)
  192 (1.92%) low severe
  5 (0.05%) low mild
  263 (2.63%) high mild
  633 (6.33%) high severe
Benchmarking readwrite/quilkin-read/254 bytes: Warming up for 3.0000 sthread 'main' panicked at benches/throughput.rs:228:36:
called `Result::unwrap()` on an `Err` value: RecvError

error: bench failed, to rerun pass `--bench throughput`

What you expected to happen:
Benchmark runs successfully.

How to reproduce it (as minimally and precisely as possible):

• Clone the latest main branch
• Run cargo bench

Anything else we need to know?:

I have rechecked the benchmark source code. My current guess is that it's something about time pauses here and where for the server warm-up. But it's just a guess and nothing more (yet).

Environment:

• Quilkin version: latest from the main branch (commit aeb2871bbfa7144cc007a10afa3300f1f6ae1815)
• Execution environment (binary, container, etc): native benchmark run without a container
• Operating system: Fedora 38, Linux kernel 6.5.6
• Custom filters? (Yes/No - if so, what do they do?): no
• Log(s): Attached above
• Others: Feel free to any additional information - I will try to provide it in the issue

[markmandel]

Just tested it on my end - and can confirm!

Not sure we've actually run the benchmarks as actual benchmarks in a long while! (or at least I haven't!) - they do get run as tests as part of CI, but not as full benchmarks.

Nice find! Will need to take a look.