Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split drop out of kafka codec benchmarks #1467

Merged
merged 2 commits into from
Feb 9, 2024

Conversation

rukai
Copy link
Member

@rukai rukai commented Feb 9, 2024

Dropping types is actually quite expensive and affects benchmark results.
In this PR:

  • To avoid measuring the drops of types we dont care about, we return them back to the criterion bencher as the return value.
  • To measure the drops of types we do care about we create a separate benchmark just for measuring the time to drop.
    • We could keep them in the same benchmark which would reduce the number of benchmarks we have but splitting them like this makes it easier to tell the costs of different operations.

decode_all bench is deleted since we no longer perform batching from within decode so it no longer makes sense as a benchmark.

Copy link

github-actions bot commented Feb 9, 2024

0 benchmark regressed. 1 benchmark improved. Please check the benchmark workflow logs for full details: https://github.com/shotover/shotover-proxy/actions/runs/7839705427

Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) low severe
  1 (1.00%) low mild
kafka_codec/encode_request_metadata
                        time:   [388.34 ns 394.84 ns 401.15 ns]
                        change: [-40.401% -31.752% -25.337%] (p = 0.00 < 0.05)
                        Performance has improved.

@rukai rukai requested a review from conorbros February 9, 2024 04:57
@rukai rukai enabled auto-merge (squash) February 9, 2024 10:52
Copy link

github-actions bot commented Feb 9, 2024

0 benchmark regressed. 2 benchmark improved. Please check the benchmark workflow logs for full details: https://github.com/shotover/shotover-proxy/actions/runs/7842754104

Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) low severe
  3 (3.00%) low mild
kafka_codec/encode_request_metadata
                        time:   [371.73 ns 378.05 ns 384.62 ns]
                        change: [-36.507% -31.704% -24.778%] (p = 0.00 < 0.05)
                        Performance has improved.
--
  1 (1.00%) low mild
  2 (2.00%) high mild
  5 (5.00%) high severe
kafka_codec/encode_request_list_offsets
                        time:   [530.83 ns 550.48 ns 573.16 ns]
                        change: [-36.879% -34.222% -31.322%] (p = 0.00 < 0.05)
                        Performance has improved.

@rukai rukai merged commit 3eb66a4 into shotover:main Feb 9, 2024
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants