bench: Add SHA256 implementation specific benchmarks #27598

hebasto · 2023-05-08T15:17:46Z

On the master branch, only the best available SHA256 implementation is being benchmarked. This PR makes bench_bitcoin benchmark all SHA256 implementations that are available on the system.

For example:

on Linux:

$ ./src/bench/bench_bitcoin -filter=SHA.*
Using the 'x86_shani(1way,2way)' SHA256 implementation

|             ns/byte |              byte/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|                1.00 |    1,002,545,462.93 |    0.4% |      0.01 | `SHA1`
|                2.91 |      344,117,991.18 |    0.1% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
|                2.21 |      453,081,794.40 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
|                2.21 |      453,396,506.58 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
|                0.53 |    1,870,520,687.49 |    0.1% |      0.01 | `SHA256 using the 'x86_shani(1way,2way)' SHA256 implementation`
|                7.90 |      126,627,134.33 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
|                3.94 |      253,850,206.07 |    0.0% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
|                1.40 |      716,247,553.38 |    0.4% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
|                1.26 |      792,706,270.13 |    0.9% |      0.01 | `SHA256D64_1024 using the 'x86_shani(1way,2way)' SHA256 implementation`
|                6.75 |      148,172,097.64 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
|                4.90 |      204,156,289.96 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way)' SHA256 implementation`
|                4.90 |      204,101,274.22 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
|                1.70 |      589,052,595.35 |    0.4% |      0.01 | `SHA256_32b using the 'x86_shani(1way,2way)' SHA256 implementation`
|                2.21 |      453,441,736.14 |    1.0% |      0.02 | `SHA3_256_1M`
|                1.92 |      521,807,101.48 |    1.0% |      0.02 | `SHA512`

on macOS (M1):

% ./src/bench/bench_bitcoin -filter=SHA.\*
Using the 'arm_shani(1way,2way)' SHA256 implementation

|             ns/byte |              byte/s |    err% |     total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
|                1.36 |      737,644,274.00 |    0.6% |      0.02 | `SHA1`
|                3.08 |      324,556,777.15 |    0.2% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
|                0.45 |    2,198,104,135.18 |    0.3% |      0.01 | `SHA256 using the 'arm_shani(1way,2way)' SHA256 implementation`
|                8.84 |      113,131,299.18 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
|                0.94 |    1,059,406,239.36 |    0.0% |      0.01 | `SHA256D64_1024 using the 'arm_shani(1way,2way)' SHA256 implementation`
|                6.17 |      162,050,659.51 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
|                1.15 |      866,637,155.98 |    0.0% |      0.01 | `SHA256_32b using the 'arm_shani(1way,2way)' SHA256 implementation`
|                1.69 |      592,636,491.59 |    0.2% |      0.02 | `SHA3_256_1M`
|                1.89 |      528,785,775.66 |    0.0% |      0.02 | `SHA512`

Found it useful, while working on #24773.

DrahtBot · 2023-05-08T15:17:50Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	sipa, martinus, MarcoFalke
Concept ACK	luke-jr

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

maflcko · 2023-05-08T15:20:22Z

What about benchmarking always all that are available on the system?

hebasto · 2023-05-08T15:22:00Z

What about benchmarking all that are available on the system?

You mean, as separated benchmarks? Without -sha-implementation option?

hebasto · 2023-05-09T16:22:09Z

Reworked according to @MarcoFalke's suggestion.

Friendly ping @martinus :)

src/bench/bench.cpp

src/bench/bench_bitcoin.cpp

hebasto · 2023-05-16T16:28:15Z

Reworked according to @martinus's comments.

hebasto · 2023-05-18T11:06:23Z

Rebased on top of the merged #27615 as it was relevant.

luke-jr · 2023-06-23T16:50:15Z

Concept ACK

hebasto · 2023-09-20T20:26:28Z

Rebased for fresh CI simultaneously with #24773 that is based on this PR.

sipa

ACK ce6df7d

My output (Ryzen 5950X):

ns/byte	byte/s	err%	total	benchmark
1.29	776,698,179.43	0.3%	0.01	`SHA256D64_1024_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
0.67	1,492,914,034.43	0.1%	0.01	`SHA256D64_1024_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
3.45	290,161,790.35	0.3%	0.01	`SHA256D64_1024_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
6.96	143,610,645.45	0.2%	0.01	`SHA256D64_1024_STANDARD using the 'standard' SHA256 implementation`
4.51	221,653,283.35	0.1%	0.01	`SHA256_32b_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
1.25	799,826,326.07	0.2%	0.01	`SHA256_32b_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
4.54	220,365,747.83	0.1%	0.01	`SHA256_32b_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
5.77	173,257,910.52	0.1%	0.01	`SHA256_32b_STANDARD using the 'standard' SHA256 implementation`
2.01	498,088,336.96	0.2%	0.02	`SHA256_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
0.44	2,288,837,568.06	0.4%	0.01	`SHA256_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
1.99	501,633,066.45	0.5%	0.02	`SHA256_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
2.50	400,467,265.21	0.2%	0.03	`SHA256_STANDARD using the 'standard' SHA256 implementation`

hebasto · 2023-10-02T11:23:44Z

@martinus

Mind taking another look at this PR?

martinus · 2023-10-02T17:52:53Z

ACK ce6df7d. I would have created a helper function in the test to avoid the code duplication for each test, but that's just me nitpicking. Here are results from my Ryzen 7950X, with ./src/bench/bench_bitcoin -filter="SHA256.*" -min-time=1000:

ns/byte	byte/s	err%	ins/byte	bra/byte	miss%	total	benchmark
1.08	922,442,557.83	0.0%	18.39	0.01	0.3%	1.07	`SHA256D64_1024_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
0.59	1,696,233,239.51	0.1%	6.60	0.03	0.1%	1.10	`SHA256D64_1024_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
2.94	339,763,959.72	0.0%	51.98	0.02	0.1%	1.10	`SHA256D64_1024_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
5.79	172,580,960.50	0.1%	132.83	0.06	0.1%	1.10	`SHA256D64_1024_STANDARD using the 'standard' SHA256 implementation`
3.82	261,777,995.08	0.1%	82.72	1.69	0.0%	1.10	`SHA256_32b_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
1.10	912,755,631.26	0.0%	14.09	1.53	0.0%	1.10	`SHA256_32b_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
3.82	261,543,369.48	0.1%	82.72	1.69	0.0%	1.10	`SHA256_32b_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
4.10	244,096,616.72	0.2%	112.94	1.53	0.0%	1.10	`SHA256_32b_STANDARD using the 'standard' SHA256 implementation`
1.75	572,561,916.85	0.1%	36.80	0.09	0.0%	1.10	`SHA256_AVX2 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
0.38	2,639,518,254.68	0.0%	2.56	0.02	0.0%	1.10	`SHA256_SHANI using the 'x86_shani(1way,2way)' SHA256 implementation`
1.75	572,393,692.61	0.2%	36.80	0.09	0.0%	1.10	`SHA256_SSE4 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
1.82	548,938,688.20	0.1%	51.91	0.02	0.0%	1.10	`SHA256_STANDARD using the 'standard' SHA256 implementation`

maflcko · 2023-10-04T13:53:20Z

review ACK ce6df7d 🏵

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: review ACK ce6df7df9bab2405cfe7d6e382f5682cf30de476 🏵
GsNRGSniZroT9qJHZFN7xnASprLA9scBcdwWOood4TqNZX7pGDvXRFuAI0buRTCEQMVV9OT5ZVnJMymXcWDPCw==

…arks ce6df7d bench: Add SHA256 implementation specific benchmarks (Hennadii Stepanov) 5f72417 Add ability to specify SHA256 implementation for benchmark purposes (Hennadii Stepanov) Pull request description: On the master branch, only the best available `SHA256` implementation is being benchmarked. This PR makes `bench_bitcoin` benchmark all `SHA256` implementations that are available on the system. For example: - on Linux: ``` $ ./src/bench/bench_bitcoin -filter=SHA.* Using the 'x86_shani(1way,2way)' SHA256 implementation | ns/byte | byte/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.00 | 1,002,545,462.93 | 0.4% | 0.01 | `SHA1` | 2.91 | 344,117,991.18 | 0.1% | 0.03 | `SHA256 using the 'standard' SHA256 implementation` | 2.21 | 453,081,794.40 | 0.1% | 0.02 | `SHA256 using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 2.21 | 453,396,506.58 | 0.1% | 0.02 | `SHA256 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 0.53 | 1,870,520,687.49 | 0.1% | 0.01 | `SHA256 using the 'x86_shani(1way,2way)' SHA256 implementation` | 7.90 | 126,627,134.33 | 0.0% | 0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation` | 3.94 | 253,850,206.07 | 0.0% | 0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 1.40 | 716,247,553.38 | 0.4% | 0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 1.26 | 792,706,270.13 | 0.9% | 0.01 | `SHA256D64_1024 using the 'x86_shani(1way,2way)' SHA256 implementation` | 6.75 | 148,172,097.64 | 0.2% | 0.01 | `SHA256_32b using the 'standard' SHA256 implementation` | 4.90 | 204,156,289.96 | 0.1% | 0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way)' SHA256 implementation` | 4.90 | 204,101,274.22 | 0.1% | 0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation` | 1.70 | 589,052,595.35 | 0.4% | 0.01 | `SHA256_32b using the 'x86_shani(1way,2way)' SHA256 implementation` | 2.21 | 453,441,736.14 | 1.0% | 0.02 | `SHA3_256_1M` | 1.92 | 521,807,101.48 | 1.0% | 0.02 | `SHA512` ``` - on macOS (M1): ``` % ./src/bench/bench_bitcoin -filter=SHA.\* Using the 'arm_shani(1way,2way)' SHA256 implementation | ns/byte | byte/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1.36 | 737,644,274.00 | 0.6% | 0.02 | `SHA1` | 3.08 | 324,556,777.15 | 0.2% | 0.03 | `SHA256 using the 'standard' SHA256 implementation` | 0.45 | 2,198,104,135.18 | 0.3% | 0.01 | `SHA256 using the 'arm_shani(1way,2way)' SHA256 implementation` | 8.84 | 113,131,299.18 | 0.0% | 0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation` | 0.94 | 1,059,406,239.36 | 0.0% | 0.01 | `SHA256D64_1024 using the 'arm_shani(1way,2way)' SHA256 implementation` | 6.17 | 162,050,659.51 | 0.2% | 0.01 | `SHA256_32b using the 'standard' SHA256 implementation` | 1.15 | 866,637,155.98 | 0.0% | 0.01 | `SHA256_32b using the 'arm_shani(1way,2way)' SHA256 implementation` | 1.69 | 592,636,491.59 | 0.2% | 0.02 | `SHA3_256_1M` | 1.89 | 528,785,775.66 | 0.0% | 0.02 | `SHA512` ``` Found it useful, while working on bitcoin#24773. ACKs for top commit: martinus: ACK ce6df7d. I would have created a helper function in the test to avoid the code duplication for each test, but that's just me nitpicking. Here are results from my Ryzen 7950X, with `./src/bench/bench_bitcoin -filter="SHA256.*" -min-time=1000`: MarcoFalke: review ACK ce6df7d 🏵 sipa: ACK ce6df7d Tree-SHA512: e3de50e11b9a3a0d1e05583786041d4dc9afa2022e2115d75d6d1f63b11f62f6336f093001e53a631431d558c4dae29c596755c9e2d6aa78c382270116cc1f7f

DrahtBot added the Tests label May 8, 2023

hebasto force-pushed the 230508-bench branch from fce8cbf to 62445aa Compare May 9, 2023 14:18

hebasto changed the title ~~bench: Add -sha-implementation command-line option~~ bench: Benchmark all SHA256 implementations that are available on the system May 9, 2023

hebasto force-pushed the 230508-bench branch from 62445aa to e5ab4a8 Compare May 9, 2023 16:19

DrahtBot mentioned this pull request May 9, 2023

build: Check usages of #if defined(...) #25302

Closed

hebasto mentioned this pull request May 10, 2023

Enable HW-accelerated implementations of SHA256 for MSVC builds #24773

Closed

martinus reviewed May 11, 2023

View reviewed changes

src/bench/bench.cpp Outdated Show resolved Hide resolved

src/bench/bench_bitcoin.cpp Outdated Show resolved Hide resolved

hebasto force-pushed the 230508-bench branch from e5ab4a8 to 4c46938 Compare May 16, 2023 16:26

hebasto changed the title ~~bench: Benchmark all SHA256 implementations that are available on the system~~ bench: Add SHA256 implementation specific benchmarks May 16, 2023

hebasto force-pushed the 230508-bench branch from 4c46938 to f3b3fbb Compare May 18, 2023 11:04

achow101 requested review from sipa and josibake September 20, 2023 16:52

hebasto added 2 commits September 20, 2023 21:11

Add ability to specify SHA256 implementation for benchmark purposes

5f72417

bench: Add SHA256 implementation specific benchmarks

ce6df7d

hebasto force-pushed the 230508-bench branch from f3b3fbb to ce6df7d Compare September 20, 2023 20:24

DrahtBot added CI failed and removed CI failed labels Sep 21, 2023

hebasto mentioned this pull request Sep 24, 2023

Add 1-way SSE4 SHA256 implementation using intrinsics for MSVC builds #28526

Closed

sipa reviewed Sep 29, 2023

View reviewed changes

martinus approved these changes Oct 2, 2023

View reviewed changes

hebasto requested a review from maflcko October 4, 2023 10:08

DrahtBot removed the request for review from maflcko October 4, 2023 13:53

fanquake merged commit 0584882 into bitcoin:master Oct 4, 2023

hebasto deleted the 230508-bench branch October 4, 2023 15:06

bitcoin locked and limited conversation to collaborators Oct 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bench: Add SHA256 implementation specific benchmarks #27598

bench: Add SHA256 implementation specific benchmarks #27598

hebasto commented May 8, 2023 •

edited

Loading

DrahtBot commented May 8, 2023 •

edited

Loading

maflcko commented May 8, 2023 •

edited

Loading

hebasto commented May 8, 2023

hebasto commented May 9, 2023

hebasto commented May 16, 2023 •

edited

Loading

hebasto commented May 18, 2023

luke-jr commented Jun 23, 2023

hebasto commented Sep 20, 2023

sipa left a comment

hebasto commented Oct 2, 2023

martinus commented Oct 2, 2023

maflcko commented Oct 4, 2023

bench: Add SHA256 implementation specific benchmarks #27598

bench: Add SHA256 implementation specific benchmarks #27598

Conversation

hebasto commented May 8, 2023 • edited Loading

DrahtBot commented May 8, 2023 • edited Loading

Reviews

Conflicts

maflcko commented May 8, 2023 • edited Loading

hebasto commented May 8, 2023

hebasto commented May 9, 2023

hebasto commented May 16, 2023 • edited Loading

hebasto commented May 18, 2023

luke-jr commented Jun 23, 2023

hebasto commented Sep 20, 2023

sipa left a comment

Choose a reason for hiding this comment

hebasto commented Oct 2, 2023

martinus commented Oct 2, 2023

maflcko commented Oct 4, 2023

hebasto commented May 8, 2023 •

edited

Loading

DrahtBot commented May 8, 2023 •

edited

Loading

maflcko commented May 8, 2023 •

edited

Loading

hebasto commented May 16, 2023 •

edited

Loading