Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarking tools #177

Merged
merged 10 commits into from
Sep 19, 2022
Merged

Benchmarking tools #177

merged 10 commits into from
Sep 19, 2022

Conversation

sergefdrv
Copy link
Contributor

This PR resolves #174.

Copy link
Contributor

@matejpavlovic matejpavlovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! 👍

cmd/bench/cmd/root.go Outdated Show resolved Hide resolved
@xosmig
Copy link
Contributor

xosmig commented Aug 8, 2022

@sergefdrv you'll need to rebase again if you want a green checkmark from the linter.

Copy link
Contributor

@xosmig xosmig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to do a more general benchmarking library for mir and then just instantiate it here for ISS?

cmd/bench/cmd/root.go Outdated Show resolved Hide resolved
cmd/bench/cmd/root.go Show resolved Hide resolved
cmd/bench/cmd/node.go Outdated Show resolved Hide resolved
cmd/bench/cmd/node.go Outdated Show resolved Hide resolved
go.mod Show resolved Hide resolved
@sergefdrv
Copy link
Contributor Author

Would it be possible to do a more general benchmarking library for mir and then just instantiate it here for ISS?

I think it would make more sense once we have another atomic broadcast protocol to benchmakr.

@sergefdrv sergefdrv force-pushed the bench branch 2 times, most recently from b015ab5 to 28c3aef Compare August 8, 2022 19:23
@sergefdrv
Copy link
Contributor Author

@xosmig since I'm on vacation, could you please merge this PR if you're fine with how I address your comments?

@xosmig
Copy link
Contributor

xosmig commented Aug 8, 2022

I think it would make more sense once we have another atomic broadcast protocol to benchmark.

We will probably need to benchmark the availability layer soon (in fact, it could even make sense to do it now, with multisigcollector).

@xosmig
Copy link
Contributor

xosmig commented Aug 8, 2022

@xosmig since I'm on vacation

Enjoy your vacation! :)

could you please merge this PR if you're fine with how I address your comments?

let's return to discussing this PR after you come back.

@matejpavlovic
Copy link
Contributor

matejpavlovic commented Aug 8, 2022

Hi! I think this PR is a pretty good start for the benchmarking tooling. Of course, a lot of things can be extended and generalized. When we have some cycles to actually use it before Sergey comes back from vacation, we can see whether that's reason enough to merge it before generalization (or even generalize it ourselves), or continue improving it as a PR and only merge when more general.
Have a great vacation @sergefdrv !

@sergefdrv
Copy link
Contributor Author

I will update the PR to use the new “SMR system” abstraction.

Sergey Fedorov and others added 7 commits September 19, 2022 15:21
Nodes gather the following statistics as they run:
 - the number of delivered requests,
 - the average number of delivered requests per second,
 - the average latency to deliver a request.

Nodes periodically output statistic records in CSV format to the
specified file or to the standard output.
Signed-off-by: Matej Pavlovic <matopavlovic@gmail.com>
Signed-off-by: Matej Pavlovic <matopavlovic@gmail.com>
@sergefdrv sergefdrv merged commit e3da585 into consensus-shipyard:main Sep 19, 2022
@xosmig xosmig removed their assignment Sep 19, 2022
@sergefdrv sergefdrv deleted the bench branch September 19, 2022 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Load generator for performance evaluation
3 participants