Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2025 benchmark update #2024

Open
Kobzol opened this issue Dec 19, 2024 · 1 comment
Open

2025 benchmark update #2024

Kobzol opened this issue Dec 19, 2024 · 1 comment

Comments

@Kobzol
Copy link
Contributor

Kobzol commented Dec 19, 2024

In 2025, it will be time to refresh and update our benchmarks again, based on our benchmark update policy.

We expect to start the update in spring 2025.

The update policy, and a guide of how to actually update the benchmarks is described here. In short:

  1. Download the new benchmark
$ cargo run --bin collector download -c Primary -a <library/binary> crate <name> <version>
  1. Commit
  2. Copy perf-config.json and .patch files from the old version
  3. Add [workspace] to Cargo.toml if it's not already there
  4. Modify collector/compile-benchmarks/README.md
  5. Modify collector/compile-benchmarks/REUSE.toml
  6. Commit
  7. Run timings in old and new version and post the results to the PR description
time -p CARGO_INCREMENTAL=0 cargo check ; cargo clean
time -p CARGO_INCREMENTAL=0 cargo build ; cargo clean
time -p CARGO_INCREMENTAL=0 cargo build --release ; cargo clean
@nnethercote
Copy link
Contributor

As per the discussion on Zulip, we're going to wait a little bit and get some ducks lined up in a row before starting on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants