Add val option for mtv requiring sim tracks matched to seeds #229
Conversation
|
Oof, this is not so nice: http://areinsvo.web.cern.ch/areinsvo/MkFit/Benchmarks/July2019_AddValOpt/SIMVAL_MTV-require-seeds/SKL-SP_CMSSW_TTbar_PU70_eff_eta_build_pt0p0_SIMVAL.png thanks for making this PR! |
|
Benchmarks + validation look good, code looks good as well. +1 from me |
But (as I'm sure you're aware, @kmcdermo) at least it's obvious that the "not so nice" part is coming almost entirely from low pT, so it's a known issue: http://areinsvo.web.cern.ch/areinsvo/MkFit/Benchmarks/July2019_AddValOpt/SIMVAL_MTV-require-seeds/SKL-SP_CMSSW_TTbar_PU70_eff_eta_build_pt2p0_SIMVAL.png |
|
Indeed, it is the usual issue, and we can still see that we are losing out
ever so slightly even at high pt in the transition region, which again is a
known issue.
The pt > 0 plot in particular just makes it look somewhere worse.
…On Wed, Jul 17, 2019 at 11:37 AM srlantz ***@***.***> wrote:
Oof, this is not so nice:
http://areinsvo.web.cern.ch/areinsvo/MkFit/Benchmarks/July2019_AddValOpt/SIMVAL_MTV-require-seeds/SKL-SP_CMSSW_TTbar_PU70_eff_eta_build_pt0p0_SIMVAL.png
thanks for making this PR!
But (as I'm sure you're aware, @kmcdermo <https://github.com/kmcdermo>)
at least it's obvious that the "not so nice" part is coming almost entirely
from low pT, so it's a known issue:
http://areinsvo.web.cern.ch/areinsvo/MkFit/Benchmarks/July2019_AddValOpt/SIMVAL_MTV-require-seeds/SKL-SP_CMSSW_TTbar_PU70_eff_eta_build_pt2p0_SIMVAL.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#229>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AB2KWJPMDUBYMZKQSHH5E4LP75DFZANCNFSM4IER7DPA>
.
|
|
@areinsvo Should we assume your benchmarks were built and run with the new default compiler on the UCSD machines, which would be icc 19.0.4? If so, it would be interesting to compare them with the benchmarks that @osschar recently ran based on that compiler (though I don't know where he posted the results) |
|
yes, this must have been with 19.0.4 ... I just posted the devel benchmark here: |
|
blak, typo in major version, moved it to proper name: |
is there a reference for this as well? |
|
well, the plots from the last PR ... there was no other change than compiler version. |
|
it looks like there was a factor of 2 speedup at high MT or in speedup units is the last PR test done with a different number of events? |
|
i didn't change anything -- that was sort of the point of this benchmark :) |
Is there a way to rerun the baseline/reference now? |
|
TBB went from the 2019.0 release to 2019.6. I don't see anything obvious that would improve our scaling so much. |
|
@slava77 - The older icc 19.0.0 is still present on phi3. Even though it's not the one you get when you run the default compilervars.sh, you can choose it preferentially over the newer 19.0.4 by issuing the following command:
The interesting aspect of the speedup you noticed is that the single-thread timing result is unchanged, but all other points on the curve apparently improved by a factor varying from 1.1x faster (at 2 threads) to 1.6x faster (at 64 threads). |
|
going back to #228 http://xrd-cache-1.t2.ucsd.edu/matevz/PKF/208-Optional-HitSorting-Prefetch-Cleanup/ , it looks like the results there are very similar to the "baseline" from #227 already shared earlier. |
|
Given the discussion that the speedup does not appear anomalous, I propose to merge these changes. If we want to rerun the benchmarks with the old vs. new compiler, I can do that as part of Issue #230. |
Using --mtv-require-seeds, you can now get MTV-like validation but with the additional requirements that the sim tracks are matched to a seed. All of the benchmark and validation plots are here, including the SIMVAL for the extra option and the normal MTV-like validation.
http://areinsvo.web.cern.ch/areinsvo/MkFit/Benchmarks/July2019_AddValOpt/