Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Switch to the blake2b_simd crate in sp-core-hashing #13548

Merged

Conversation

koute
Copy link
Contributor

@koute koute commented Mar 6, 2023

It looks like switching from the blake2 crate to blake2b_simd speeds up BLAKE2 hashing by at least ~20%.

fix size hashing/blake2_128
                        time:   [112.01 ns 112.16 ns 112.33 ns]
                        change: [-24.471% -24.339% -24.210%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/2
                        time:   [114.94 ns 114.98 ns 115.03 ns]
                        change: [-24.708% -24.611% -24.517%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/6
                        time:   [115.79 ns 115.84 ns 115.90 ns]
                        change: [-21.694% -21.571% -21.459%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/10
                        time:   [116.65 ns 116.75 ns 116.84 ns]
                        change: [-22.769% -22.679% -22.594%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/14
                        time:   [113.47 ns 113.54 ns 113.61 ns]
                        change: [-22.936% -22.846% -22.758%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/18
                        time:   [115.23 ns 115.34 ns 115.45 ns]
                        change: [-23.221% -23.088% -22.955%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/22
                        time:   [115.08 ns 115.16 ns 115.25 ns]
                        change: [-25.066% -24.937% -24.808%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/26
                        time:   [117.66 ns 117.69 ns 117.72 ns]
                        change: [-21.001% -20.837% -20.674%] (p = 0.00 < 0.05)
                        Performance has improved.
dyn size hashing/blake2_128/30
                        time:   [117.93 ns 117.98 ns 118.02 ns]
                        change: [-21.243% -21.075% -20.915%] (p = 0.00 < 0.05)
                        Performance has improved.

I also checked whether enabling the simd feature on the original blake2 crate helps, and that made the benchmarks slower.

@koute koute added A0-please_review Pull request needs code review. I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit labels Mar 6, 2023
@koute koute requested a review from a team March 6, 2023 14:36
@dmitry-markin dmitry-markin requested a review from a team March 6, 2023 14:40
@ggwpez
Copy link
Member

ggwpez commented Mar 6, 2023

bot bench $ pallet dev frame_benchmarking

There was also the idea for Blake3 #12785

@command-bot
Copy link

command-bot bot commented Mar 6, 2023

@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2490171 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev frame_benchmarking. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 24-aec9b830-1dab-4ee4-b8ad-2c2ac13164e9 to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Mar 6, 2023

@ggwpez Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev frame_benchmarking has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2490171 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2490171/artifacts/download.

@koute
Copy link
Contributor Author

koute commented Mar 7, 2023

Looks like the weight benchmarks also see a minor improvement/stay mostly the same (21.9us -> 21.4us). Those benches only hash 4 bytes though, so we can't expect miracles. @ggwpez Maybe we should add a few more benches there which try to hash more data?

@ggwpez
Copy link
Member

ggwpez commented Mar 7, 2023

Looks like the weight benchmarks also see a minor improvement/stay mostly the same (21.9us -> 21.4us). Those benches only hash 4 bytes though, so we can't expect miracles. @ggwpez Maybe we should add a few more benches there which try to hash more data?

You can add some more, but in most cases we only hash AccountIDs so the 4 byte is a good baseline.

@koute
Copy link
Contributor Author

koute commented Mar 7, 2023

bot merge

@paritytech-processbot paritytech-processbot bot merged commit 6a15476 into paritytech:master Mar 7, 2023
btwiuse pushed a commit to btwiuse/substrate-benchmark-machine that referenced this pull request Mar 27, 2023
ukint-vs pushed a commit to gear-tech/substrate that referenced this pull request Apr 10, 2023
…3548)

* Switch to the `blake2b_simd` crate in `sp-core-hashing`

* ".git/.scripts/commands/bench/bench.sh" pallet dev frame_benchmarking

---------

Co-authored-by: command-bot <>
nathanwhit pushed a commit to nathanwhit/substrate that referenced this pull request Jul 19, 2023
…3548)

* Switch to the `blake2b_simd` crate in `sp-core-hashing`

* ".git/.scripts/commands/bench/bench.sh" pallet dev frame_benchmarking

---------

Co-authored-by: command-bot <>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task.
Projects
Status: done
Development

Successfully merging this pull request may close these issues.

4 participants