Benchmarks: report tx/s as elements/s #345
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Based on my limited understanding of Criterion, I think we can get add transactions per second reporting to benchmarks using Criterion's Throughput::Elements tracking, by treating transactions as elements. This is not ideal, as the output says e.g. `thrpt: [38.763 Kelem/s 38.781 Kelem/s 38.799 Kelem/s]`, where we'd like `Ktx/s`, but it's not clear whether Criterion allows us that much customization. Each of the benchmarks currently does one tx per iteration, so pass `Throughput::Elements(1)` for each of them. Do this as close to the test as possible, despite redundancy, in case we ever add benchmarks to the same group which do multiple transactions per iteration.
Contributor
|
benchmarks please |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
Based on my limited understanding of Criterion, I think we can get add transactions per second reporting to benchmarks using Criterion's Throughput::Elements tracking, by treating transactions as elements.
This is not ideal, as the output says e.g.
thrpt: [38.763 Kelem/s 38.781 Kelem/s 38.799 Kelem/s], where we'd likeKtx/s, but it's not clear whether Criterion allows us that much customization.Each of the benchmarks currently does one tx per iteration, so pass
Throughput::Elements(1)for each of them. Do this as close to the test as possible, despite redundancy, in case we ever add benchmarks to the same group which do multiple transactions per iteration.API and ABI
If the API is breaking, please state below what will break