Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-add "run all" mode to performance tester #272

Merged
merged 1 commit into from
Jan 11, 2021
Merged

Re-add "run all" mode to performance tester #272

merged 1 commit into from
Jan 11, 2021

Conversation

Lukasa
Copy link
Contributor

@Lukasa Lukasa commented Jan 8, 2021

The run-all mode of the performance tester inadvertently got removed in
a previous patch. This adds it back.

@Lukasa Lukasa added the 🔨 semver/patch No public API change. label Jan 8, 2021
@Lukasa Lukasa requested a review from weissi January 8, 2021 11:31
@Lukasa
Copy link
Contributor Author

Lukasa commented Jan 8, 2021

@swift-nio-bot test perf please

@swift-server-bot
Copy link

performance report

build id: 6

timestamp: Fri Jan 8 11:34:13 UTC 2021

results

nameminmaxmeanstd
1_conn_10k_reqs 2.002050995826721 2.0118240118026733 2.007257413864136 0.002686396425603722
encode_100k_header_blocks_indexable 0.16892099380493164 0.1701369285583496 0.16943279504776002 0.0003462860176126182
encode_100k_header_blocks_nonindexable 0.29805004596710205 0.29895591735839844 0.2986179947853088 0.0002932664717398542
encode_100k_header_blocks_neverIndexed 0.30057501792907715 0.30143702030181885 0.30101540088653567 0.0003445496123613074
decode_100k_header_blocks_indexable 0.1935499906539917 0.1957099437713623 0.1943560004234314 0.0006337048203074109
decode_100k_header_blocks_nonindexable 0.26665401458740234 0.27337002754211426 0.26749061346054076 0.0020715356929899855
decode_100k_header_blocks_neverIndexed 0.2667809724807739 0.2682300806045532 0.2670720100402832 0.00043685491903890095
huffman_encode_basic 0.3059110641479492 0.3065659999847412 0.30628780126571653 0.000260023902719785
huffman_encode_complex 0.24559307098388672 0.24641001224517822 0.24594630002975465 0.00024831195147547477
huffman_decode_basic 0.06370604038238525 0.06423091888427734 0.06386830806732177 0.0002272275185241999
huffman_decode_complex 0.04831194877624512 0.048822999000549316 0.0484860897064209 0.0002307827587826966
server_only_10k_requests_1_concurrent 0.5027649402618408 0.503588080406189 0.5032833814620972 0.00024047133432210044
server_only_10k_requests_100_concurrent 0.4083629846572876 0.4090859889984131 0.40870769023895265 0.00025433885548746397
stream_teardown_10k_requests_100_concurrent 0.2278200387954712 0.22853899002075195 0.22813490629196168 0.00023385916993581013

comparison

name current previous winner diff
1_conn_10k_reqs 2.007257413864136 2.221388208866119 current -9%
encode_100k_header_blocks_indexable 0.16943279504776002 0.3241667032241821 current -47%
encode_100k_header_blocks_nonindexable 0.2986179947853088 0.290792191028595 previous 2%
encode_100k_header_blocks_neverIndexed 0.30101540088653567 0.28608070611953734 previous 5%
decode_100k_header_blocks_indexable 0.1943560004234314 0.20769050121307372 current -5%
decode_100k_header_blocks_nonindexable 0.26749061346054076 0.28266890048980714 current -5%
decode_100k_header_blocks_neverIndexed 0.2670720100402832 0.28234760761260985 current -5%
huffman_encode_basic 0.30628780126571653 0.3034381031990051 previous 0%
huffman_encode_complex 0.24594630002975465 0.2507891058921814 current -1%
huffman_decode_basic 0.06386830806732177 0.0703089952468872 current -9%
huffman_decode_complex 0.0484860897064209 0.05351079702377319 current -9%
server_only_10k_requests_1_concurrent 0.5032833814620972 n/a n/a%
server_only_10k_requests_100_concurrent 0.40870769023895265 n/a n/a%
stream_teardown_10k_requests_100_concurrent 0.22813490629196168 n/a n/a%

significant differences found

@Lukasa Lukasa force-pushed the cb-performance-tester-run-all branch 4 times, most recently from fa55ea6 to 0aea264 Compare January 11, 2021 07:23
@Lukasa
Re-add "run all" mode to performance tester
292dce6 
The run-all mode of the performance tester inadvertently got removed in
a previous patch. This adds it back.

Along the way we have apparently unlocked some perf gains in a new NIO
release.
@Lukasa Lukasa force-pushed the cb-performance-tester-run-all branch from 0aea264 to 292dce6 Compare January 11, 2021 07:37
@Lukasa Lukasa requested a review from weissi January 11, 2021 09:06
@Lukasa
Copy link
Contributor Author

Lukasa commented Jan 11, 2021

@weissi A new NIO release knocked some allocs off, so I'm asking for a re-review from you.

@weissi weissi merged commit 59941f6 into apple:main Jan 11, 2021
@Lukasa Lukasa deleted the cb-performance-tester-run-all branch January 11, 2021 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weissi weissi weissi approved these changes

Assignees
No one assigned
Labels
🔨 semver/patch No public API change.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Lukasa @swift-server-bot @weissi