Generate stats from loadgen run #36
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Nov 24, 2021
|
Hm, does that mean if something interrupts the run, we get no stats data? That sounds kinda fragile. I was expecting a tool that executes after the run is complete and derives stats from the logs. |
|
Unless the runner crashes in some way I haven't anticipated, it should always generate summaries for the stages and the run, and output stats as |
mhofman
force-pushed
the
mhofman/preliminary-runner-refactors
branch
from
3050fd7 to
0348471
Compare
mhofman
force-pushed
the
mhofman/add-stats
branch
from
4d96e29 to
541e6a8
Compare
Various fixes to loadgen and runner Fixes #28. Handles rename of `Treasury` -> `VaultFactory` Handle older SDK versions which don't have a fee purse implementation
warner
approved these changes
Jan 17, 2022
mhofman
force-pushed
the
mhofman/add-stats
branch
from
541e6a8 to
6ff64f6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is crux of the stats work.
After multiple rewrites, I settled on stats objects passed to individual monitors, with methods on the stats objects to capture important events and trigger computation of derived stats. Stats objects expose read-only properties (either getters or non-writable values), and form a tree through nested collections of stats objects. The root
statsis serialized to JSON at the end of the loadgen run, and saved in thefinishperf event saved to disk.Currently only block and cycle stats per stage, as well as their summaries, are generated. Process and storage usage stats can be added later.
run->stages->blocksandcyclesIn the future we can also consider saving and uploading intermediate states of the stats, e.g. after each stage completes, or even as soon as each block completes if we want to build a real-time dashboard.
As always, best reviewed commit-by-commit.
Stacks on top of preliminary refactors (#35) and fixes(#34).