Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify blake3 hasher and improve performance #19041

Closed

Conversation

tylerwilliams
Copy link
Contributor

@tylerwilliams tylerwilliams commented Jul 24, 2023

Improve BLAKE3 performance by reducing copies.

Rather than buffering bytes, it's faster to just update the blake3 hasher as bytes are hashed.

Performance with buffering:

 Benchmark                   (inputSize)  Mode  Cnt      Score     Error  Units
 Blake3Benchmark.blake3Hash            1  avgt   10   1766.697 ± 709.515  ns/op
 Blake3Benchmark.blake3Hash           10  avgt   10   1466.253 ±  19.474  ns/op
 Blake3Benchmark.blake3Hash          100  avgt   10   1522.845 ±  15.480  ns/op
 Blake3Benchmark.blake3Hash         1000  avgt   10   2254.156 ±   8.588  ns/op
 Blake3Benchmark.blake3Hash        10000  avgt   10   4660.881 ±  28.637  ns/op
 Blake3Benchmark.blake3Hash       100000  avgt   10  24283.191 ±  32.754  ns/op
 Blake3Benchmark.sha256Hash            1  avgt   10    742.091 ±   6.307  ns/op
 Blake3Benchmark.sha256Hash           10  avgt   10    757.844 ±  12.042  ns/op
 Blake3Benchmark.sha256Hash          100  avgt   10    942.902 ± 555.874  ns/op
 Blake3Benchmark.sha256Hash         1000  avgt   10   1208.336 ± 508.392  ns/op
 Blake3Benchmark.sha256Hash        10000  avgt   10   4871.231 ± 494.507  ns/op
 Blake3Benchmark.sha256Hash       100000  avgt   10  40686.231 ±  63.814  ns/op

Performance without buffering (after this CL):

 Benchmark                   (inputSize)  Mode  Cnt      Score     Error  Units
 Blake3Benchmark.blake3Hash            1  avgt   10   1021.075 ±  11.640  ns/op
 Blake3Benchmark.blake3Hash           10  avgt   10   1029.561 ±  19.850  ns/op
 Blake3Benchmark.blake3Hash          100  avgt   10   1070.509 ±  12.140  ns/op
 Blake3Benchmark.blake3Hash         1000  avgt   10   1685.043 ±  13.963  ns/op
 Blake3Benchmark.blake3Hash        10000  avgt   10   3939.516 ±  13.212  ns/op
 Blake3Benchmark.blake3Hash       100000  avgt   10  21730.550 ±  22.976  ns/op
 Blake3Benchmark.sha256Hash            1  avgt   10    745.943 ±   9.853  ns/op
 Blake3Benchmark.sha256Hash           10  avgt   10    747.649 ±  17.381  ns/op
 Blake3Benchmark.sha256Hash          100  avgt   10    962.802 ± 590.879  ns/op
 Blake3Benchmark.sha256Hash         1000  avgt   10   1189.069 ± 412.327  ns/op
 Blake3Benchmark.sha256Hash        10000  avgt   10   4594.978 ±  21.833  ns/op
 Blake3Benchmark.sha256Hash       100000  avgt   10  39224.248 ± 229.265  ns/op

@github-actions github-actions bot added the awaiting-review PR is awaiting review from an assigned reviewer label Jul 24, 2023
@tylerwilliams tylerwilliams force-pushed the blake3_simplify branch 2 times, most recently from fb651a5 to 3c81c52 Compare July 24, 2023 23:51
@brentleyjones
Copy link
Contributor

@coeuvre

@sgowroji sgowroji added the team-Performance Issues for Performance teams label Jul 25, 2023
Copy link
Member

@coeuvre coeuvre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I like this change not only because it improves the performance but also makes the code easier to read/maintain.

flush();
}
buffer.put(b);
engineUpdate(new byte[] {b}, 0, 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know whether creating a temporary array here will impact performance (and I have no idea when this method will be called), but can we avoid doing that? For example, we could have a 1 sized byte array buffer created ahead of time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, done.

@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Jul 26, 2023
@brentleyjones brentleyjones deleted the blake3_simplify branch August 2, 2023 16:18
@iancha1992
Copy link
Member

The changes in this PR have been included in Bazel 6.4.0 RC1. Please test out the release candidate and report any issues as soon as possible. If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=last_rc.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Performance Issues for Performance teams
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants