Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Describe somewhere what the benchmarks measure, and how to measure yourself #275

Closed
RalfJung opened this issue Aug 21, 2018 · 1 comment
Closed

Comments

@RalfJung
Copy link
Member

I just realized that "baseline" and "clean" are the exact opposite of what I thought they were. "clean" sounds like it starts with a clean (empty) cache, and "baseline" like it would be the baseline, minimal cost of an incremental run with a hot cache. But actually it's the other way around.

Could the timing pages (like this one) explain this somewhere? Like a note at the bottom, or a tooltip in the actual table, or so. Having space for such an explanation would anyway be useful, as that would also be a good space to describe how one can run that same benchmark against a local rustc -- which usually is the first thing I need to do when I see a regression.

(I am still working on figuring out how to reproduce this "incremental clean" regression. Took me at least an hour, at this point, due to running into issues like rust-lang/cargo#5918.)

@Mark-Simulacrum
Copy link
Member

Fixed in b52f402.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants