-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Per-version hashing information #1126
Comments
This was referenced Dec 20, 2021
This was referenced Dec 21, 2021
This was referenced Dec 21, 2021
This is essentially fixed by #1127. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Since the merge of rust-lang/rust#89836, we are seeing high variance on the regression-31157-opt benchmark:
This is likely due to hashing being effectively randomized, affecting some hash table layout (I've not narrowed down where/what exactly).
I suspect this may be creating noise on other benchmarks, as well. Our significance algorithm should learn to cope with this in another ~20 commits (roughly 2-3 days) as our backwards window is for 50 commits, but it likely makes sense to try to avoid this problem at the root, and in general it'll make regression triage harder by increasing the baseline changes commit to commit even with small code deltas.
I am not sure yet whether a revert of rust-lang/rust#89836 makes sense, or whether we should use RUSTC_FORCE_INCR_COMP_ARTIFACT_HEADER on rustc-perf to reduce noise levels... in the meantime I'll set the env variable for perf and that'll hopefully eliminate noise in the meantime.
cc @jyn514
The text was updated successfully, but these errors were encountered: