Switching between samples with a flag

[kmcdermo]

One possible improvement to the physics validation is a flag to switch between samples to process. The idea is to pass a single flag to the validation scripts to move between ttbar noPU, PU35, PU70, and 10mu.

Inside the validation script, the flag would set the sub directory for the samples, as well as the amount of events to process. Currently, we use 500 events for ttbar PU70, so we would need to scale accordingly for the other samples (1000 for ttbar PU35, 2000 for noPU, and 10000 events for 10mu, maybe?).

This flag would also need to handle the renaming of the output files appropriately such that the web scripts also pick up on the right labels.

Since it only really makes sense to test with ttbar PU70 for compute tests, this would not affect the benchmarking script. If we did want to test something other than PU70 in the benchmarking, we would need high stats samples of the other datasets to give enough work to the tests, specifically MEIF. We use 20 events * number of events in flight for PU70, so on KNL at full load, we are processing 2560 events. Given that the other samples have less work we would probably even need to bump up the number of events to process per event in flight.

[areinsvo]

This was added in PR218 and PR249 to switch between ttbar PU50, ttbar PU70, and the offline and online 10muon samples. The second PR included changing the number of events to 10000 if we are processing 10muon. I don't think there is any interest (or new memory files) for ttbar noPU or PU35, unless someone disagrees.

I don't think there is a need to run benchmarking with any sample other than ttbar PU50. That means the only open action item for this issue is to pass the name of the sample to the plotting scripts to update the output file names and plot labels.

[slava77]

I don't think there is a need to run benchmarking with any sample other than ttbar PU50. That means the only open action item for this issue is to pass the name of the sample to the plotting scripts to update the output file names and plot labels.

I think that this is done already by setting a value of the "sample" variable, as e.g. in d3c6eb5#diff-45494913097202319b94136f9d990897R49

Is something still not showing up appropriately?

[areinsvo]

@slava77 the missing piece is getting web/move-benchmarks.sh to pick up the "sample" variable appropriately. Right now it sources xeon_scripts/common-variables.sh, which sets sample to CMS_TTBar_PU50 here.