Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subsystem-benchmarks: estimate reference hardware CPU usage #5432

Open
sandreim opened this issue Aug 21, 2024 · 0 comments
Open

subsystem-benchmarks: estimate reference hardware CPU usage #5432

sandreim opened this issue Aug 21, 2024 · 0 comments
Assignees
Labels
T10-tests This PR/Issue is related to tests. T12-benchmarks This PR/Issue is related to benchmarking and weights.

Comments

@sandreim
Copy link
Contributor

In CI we need to run the benchmarks on ref hardware machines, but development is usually done on faster machines. I think it would be both interesting and useful to consider reporting some estimations for reference hardware.

One idea is to bundle the polkadot node benchmark in the CLI tool and run it once to get the following metrics:Blake2256, Sr25519Verify, MemCopy. We then use these metrics to compute a perf_index coefficient . Assuming we have determined a similar value for the reference hardware we can the estimate CPU usage from the results of the local development machine:
estimated_ref_hw_cpu = local_perf_index / ref_perf_index * local_cpu_usage

@sandreim sandreim added T10-tests This PR/Issue is related to tests. T12-benchmarks This PR/Issue is related to benchmarking and weights. labels Aug 21, 2024
@AndreiEres AndreiEres self-assigned this Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T10-tests This PR/Issue is related to tests. T12-benchmarks This PR/Issue is related to benchmarking and weights.
Projects
Status: Backlog
Development

No branches or pull requests

2 participants