automatic finding of edm and pure root nano branches #1872

vlimant · 2022-11-02T11:33:46Z

as commented in #1866 (comment) the current script is not parsing the edm flat table automatically, and is not compatible with pure-root nano output.
this PR tries to address both issues.
I might include the matrix_RE change for the new nano workflows

cmsbuild · 2022-11-02T11:34:11Z

A new Pull Request was created by @vlimant (vlimant) for branch master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

comparisons/validate.C

cmsbuild · 2022-11-02T12:16:28Z

Pull request #1872 was updated.

comparisons/matrix_RE.txt

…d branches

cmsbuild · 2022-11-02T15:04:13Z

Pull request #1872 was updated.

slava77 · 2022-11-04T19:22:42Z

@cmsbuild please test

I was thinking of asking for extra workflows, but I don't think the bot would generate the baseline for them.

cmsbuild · 2022-11-04T22:41:40Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6c4b56/28795/summary.html
COMMIT: befb7dc
CMSSW: CMSSW_12_6_X_2022-11-04-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/1872/28795/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3416402
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3416377
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 206 log files, 48 edm output root files, 48 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-11-05T10:18:10Z

please test

cmsbuild · 2022-11-05T13:43:23Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6c4b56/28800/summary.html
COMMIT: befb7dc
CMSSW: CMSSW_12_6_X_2022-11-04-2300/el8_amd64_gcc10
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/1872/28800/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3416402
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3416380
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 206 log files, 48 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.001_data80X+MuonEG2016HMINIAOD8.0+NANO_data8.0+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.0_mc80X+TTbarMINIAOD8.0+NANO_mc8.0+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.101_mc94Xv2+TTbarMINIAOD9.4v2+NANO_mc9.4v2+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.11_data94X2016+MuonEG2016MINIAOD9.4v2+NANO_2016data9.4v2+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.12_data94Xv2+MuonEG2017MINIAOD9.4v2+NANO_data9.4v2+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.1_mc94X2016+TTbar2016MINIAOD9.4v2+NANO_2016mc9.4v2+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.21_data102X+MuonEG2018MINIAOD10.2+NANO_data10.2+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.2_mc102X+TTbarMINIAOD10.2+NANO_mc10.2+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.301_mc106Xul17+TTbarMINIAOD10.6_UL17v1+NANO_mc10.6ul17v1+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.302_mc106Xul18+TTbarMINIAOD10.6_UL18v1+NANO_mc10.6ul18v1+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.311_mc106Xul17v2+TTbarMINIAOD10.6_UL17v2+NANO_mc10.6ul17v2+HRV_NANO_mc
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.321_data106Xul17+MuonEG2017MINIAOD10.6v1+NANO_data10.6ul17v1+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.322_data106Xul18+MuonEG2018MINIAOD10.6v1+NANO_data10.6ul18v1+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.32_data106Xul16+MuonEG2016MINIAOD10.6v1+NANO_data10.6ul16v1+HRV_NANO_data
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-6c4b56/2500.3_mc106Xul16+TTbarMINIAOD10.6_UL16v1+NANO_mc10.6ul16v1+HRV_NANO_mc

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 29
DQMHistoTests: Total histograms compared: 28879
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 28879
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 28 files compared)
Checked 66 log files, 0 edm output root files, 29 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.0	2.140	2.140	0.000 ( +0.0% )	6.77	7.02	-3.5%	1.413	1.419
2500.001	1.119	1.119	0.000 ( +0.0% )	9.86	10.22	-3.5%	1.396	1.405
2500.1	2.267	2.267	0.000 ( +0.0% )	5.13	5.30	-3.3%	1.770	1.775
2500.101	2.351	2.351	0.000 ( +0.0% )	4.35	4.47	-2.7%	1.937	1.947
2500.11	1.212	1.212	0.000 ( +0.0% )	7.99	8.17	-2.2%	1.615	1.611
2500.12	1.319	1.319	0.000 ( +0.0% )	5.80	6.04	-4.0%	1.846	1.868
2500.2	2.195	2.195	0.000 ( +0.0% )	5.08	5.28	-3.7%	1.810	1.818
2500.21	1.211	1.211	0.000 ( +0.0% )	7.59	7.80	-2.7%	1.856	1.858
2500.3	2.312	2.312	0.000 ( +0.0% )	4.94	5.10	-3.3%	1.762	1.770
2500.301	2.459	2.459	0.000 ( +0.0% )	4.50	4.63	-2.8%	1.524	1.529
2500.302	2.411	2.411	0.000 ( +0.0% )	4.62	4.75	-2.8%	1.824	1.829
2500.31	2.206	2.206	0.000 ( +0.0% )	8.30	8.72	-4.8%	1.480	1.491
2500.311	2.327	2.327	0.000 ( +0.0% )	7.99	8.26	-3.3%	1.876	1.888
2500.312	2.278	2.278	0.000 ( +0.0% )	8.10	8.38	-3.4%	1.859	1.871
2500.32	1.081	1.081	0.000 ( +0.0% )	8.56	8.96	-4.5%	1.627	1.626
2500.321	1.456	1.456	0.000 ( +0.0% )	5.77	6.05	-4.6%	1.803	1.801
2500.322	1.229	1.229	0.000 ( +0.0% )	7.34	7.43	-1.2%	1.522	1.525
2500.33	1.095	1.095	0.000 ( +0.0% )	19.45	20.01	-2.8%	1.491	1.491
2500.331	1.391	1.391	0.000 ( +0.0% )	14.44	14.77	-2.2%	1.648	1.654
2500.332	1.323	1.323	0.000 ( +0.0% )	16.17	16.40	-1.4%	1.585	1.583
2500.4	2.115	2.115	0.000 ( +0.0% )	9.40	9.57	-1.8%	1.359	1.383
2500.401	2.115	2.115	0.000 ( +0.0% )	9.43	9.62	-2.1%	1.447	1.447
2500.5	0.302	0.302	0.000 ( +0.0% )	47.19	47.71	-1.1%	1.206	1.200
2500.501	0.302	0.302	0.000 ( +0.0% )	47.18	48.11	-1.9%	1.216	1.225
2500.51	1.100	1.100	0.000 ( +0.0% )	26.90	27.37	-1.7%	1.334	1.335
2500.511	1.100	1.100	0.000 ( +0.0% )	26.54	27.72	-4.2%	1.382	1.355
2500.6	1.418	1.418	0.000 ( +0.0% )	23.43	23.47	-0.2%	1.088	1.051
2500.601	1.418	1.418	0.000 ( +0.0% )	23.98	24.00	-0.1%	1.200	1.212

comparisons/validate.C

vlimant · 2022-11-20T19:26:17Z

In file included from /afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/validate_C_ACLiC_dict.h:34:0,
from /afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/validate_C_ACLiC_dict.cxx:17:
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:33:6: warning: unused parameter 'step' [-Wunused-parameter]
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C: In function 'PlotStats plotvar(TString, TString, bool)':
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:164:3: error: 'SetExponentOffset' is not a member of 'TGaxis'
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:165:3: error: 'SetExponentOffset' is not a member of 'TGaxis'
In file included from /afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/validate_C_ACLiC_dict.h:34:0,
from /afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/validate_C_ACLiC_dict.cxx:17:
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C: At global scope:
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:1544:6: warning: unused parameter 'step' [-Wunused-parameter]
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C: In function 'void validateEvents(TString, TString, TString, TString, bool, TString)':
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:1986:34: error: range-based 'for' loops are not allowed in C++98 mode
/afs/cern.ch/work/v/vlimant/private/CMSSW_5_3_X_2022-11-20-0000/src/validate_lib/./validate.C:2848:34: error: range-based 'for' loops are not allowed in C++98 mode

those are not specific to this PR ; do we really have to go back to 5.3 ?

cmsbuild · 2022-11-21T12:17:51Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6c4b56/29154/summary.html
COMMIT: 374e203
CMSSW: CMSSW_10_6_X_2022-11-13-0000/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/1872/29154/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
4.22 step 4
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 3179233
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3178910
DQMHistoTests: Total skipped: 320
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 142 log files, 29 edm output root files, 34 DQM output files
TriggerResults: no differences found

slava77 · 2022-11-21T12:59:24Z

those are not specific to this PR ; do we really have to go back to 5.3 ?

is it newly broken or was it already broken?

vlimant · 2022-11-21T13:02:30Z

I have started a test in 5.3 in cms-sw/cmssw#40031

slava77 · 2022-11-21T14:14:49Z

I have started a test in 5.3 in cms-sw/cmssw#40031

I see there in the tests from 12 days ago

Error: Function enable() is not defined in current scope  (tmpfile):1:

so, it was broken in some way already

vlimant · 2022-11-21T20:02:41Z

indeed, https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_5_3_X_2022-11-06-0000+3600ef/54007/validateJR/validateJR.log the same compilation error is already there

slava77 · 2022-11-21T20:20:39Z

indeed, https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_5_3_X_2022-11-06-0000+3600ef/54007/validateJR/validateJR.log the same compilation error is already there

perhaps it's an excuse to not go back that far in older releases ;)

slava77 · 2022-11-21T20:23:09Z

10_6_X results look OK.

slava77 · 2022-11-21T20:24:22Z

@clacaputo @mandrenguyen
in case you haven't looked at this PR already , please check.
The format of what you are looking at frequently is going to change quite a bit

vlimant · 2022-11-22T08:37:51Z

this vlimant@fb036c7 solves the compilation error in 5.3, but it does not avoid the error coming from
FWLiteEnabler::enable() Error: Function enable() is not defined in current scope (tmpfile):1: *** Interpreter error recovered ***

smuzaffar · 2022-11-22T08:59:51Z

For 5.3.X, AutoLibraryLoader::enable() is used. We call both AutoLibraryLoader::enable() and FWLiteEnabler::enable() to make sure that lib loadig is enable for any release. I think FWLiteEnabler::enable() Error: Function enable() is not defined in current scope (tmpfile):1: *** Interpreter error recovered error should not be causing any issues.

smuzaffar · 2022-11-22T09:03:50Z

We can avoid FWLiteEnabler::enable() Error: Function enable() is not defined in current scope (tmpfile) if we conditionally add this if $CMSSW_RELEASE_BASE/rc/FWCore/FWLite/interface/FWLiteEnabler.h exists

smuzaffar · 2022-11-22T09:20:13Z

+externals
As 5.3.X errors are already there which we can fix later. We can merge this after green single from @clacaputo @mandrenguyen

cmsbuild · 2022-11-22T09:20:38Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

vlimant · 2022-11-22T10:12:09Z

that vlimant@66019a2 works and can go in a subsequent PR

clacaputo · 2022-11-22T11:23:52Z

+reconstruction

@vlimant thanks a lot for taking care of it

smuzaffar · 2022-11-22T13:08:42Z

merging it now, @vlimant feel free to open a new PR for 5.3.X fixes

vlimant · 2022-11-22T13:31:40Z

thanks !

slava77 · 2022-11-22T14:17:47Z

thanks !

Thank you.
automation is better

automatic finding of edm and pure root nano branches

b531b03

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Nov 2, 2022

vlimant mentioned this pull request Nov 2, 2022

Integrate part of NANO test in cmsbot commands cms-sw/cmssw#39452

Closed

vlimant commented Nov 2, 2022

View reviewed changes

comparisons/validate.C Outdated Show resolved Hide resolved

vlimant commented Nov 2, 2022

View reviewed changes

comparisons/validate.C Outdated Show resolved Hide resolved

add comparison to recenly added nano workflows

7c484db

slava77 reviewed Nov 2, 2022

View reviewed changes

comparisons/matrix_RE.txt Outdated Show resolved Hide resolved

slava77 reviewed Nov 2, 2022

View reviewed changes

comparisons/matrix_RE.txt Outdated Show resolved Hide resolved

vlimant added 2 commits November 2, 2022 15:58

use ref tree for list of branches ; does not work for newly untroduce…

204e827

…d branches

each in each category

befb7dc

cmsbuild added tests-started and removed tests-pending labels Nov 4, 2022

cmsbuild added tests-approved and removed tests-started labels Nov 4, 2022

cmsbuild added tests-started and removed tests-approved labels Nov 5, 2022

cmsbuild added tests-approved and removed tests-started labels Nov 5, 2022

slava77 reviewed Nov 5, 2022

View reviewed changes

comparisons/validate.C Outdated Show resolved Hide resolved

revert to previous check on step

f9289e9

cmsbuild added tests-started and removed tests-approved labels Nov 21, 2022

vlimant mentioned this pull request Nov 21, 2022

Update EDM inputs to HLT-Validation tests [5_3_X only] cms-sw/cmssw#40031

Merged

cmsbuild added tests-approved and removed tests-started labels Nov 21, 2022

cmsbuild added externals-approved fully-signed and removed pending-signatures externals-pending labels Nov 22, 2022

smuzaffar merged commit bd7f08c into cms-sw:master Nov 22, 2022

vlimant deleted the update_validate_4_nano branch November 22, 2022 13:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

automatic finding of edm and pure root nano branches #1872

automatic finding of edm and pure root nano branches #1872

vlimant commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

slava77 commented Nov 4, 2022 •

edited

Loading

cmsbuild commented Nov 4, 2022

smuzaffar commented Nov 5, 2022

cmsbuild commented Nov 5, 2022

vlimant commented Nov 20, 2022

cmsbuild commented Nov 21, 2022

slava77 commented Nov 21, 2022

vlimant commented Nov 21, 2022

slava77 commented Nov 21, 2022

vlimant commented Nov 21, 2022

slava77 commented Nov 21, 2022

slava77 commented Nov 21, 2022

slava77 commented Nov 21, 2022

vlimant commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

smuzaffar commented Nov 22, 2022 •

edited

Loading

cmsbuild commented Nov 22, 2022

vlimant commented Nov 22, 2022

clacaputo commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

vlimant commented Nov 22, 2022

slava77 commented Nov 22, 2022

automatic finding of edm and pure root nano branches #1872

automatic finding of edm and pure root nano branches #1872

Conversation

vlimant commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

cmsbuild commented Nov 2, 2022

slava77 commented Nov 4, 2022 • edited Loading

cmsbuild commented Nov 4, 2022

Comparison Summary

smuzaffar commented Nov 5, 2022

cmsbuild commented Nov 5, 2022

Comparison Summary

NANO Comparison Summary

vlimant commented Nov 20, 2022

cmsbuild commented Nov 21, 2022

Comparison Summary

slava77 commented Nov 21, 2022

vlimant commented Nov 21, 2022

slava77 commented Nov 21, 2022

vlimant commented Nov 21, 2022

slava77 commented Nov 21, 2022

slava77 commented Nov 21, 2022

slava77 commented Nov 21, 2022

vlimant commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

smuzaffar commented Nov 22, 2022 • edited Loading

cmsbuild commented Nov 22, 2022

vlimant commented Nov 22, 2022

clacaputo commented Nov 22, 2022

smuzaffar commented Nov 22, 2022

vlimant commented Nov 22, 2022

slava77 commented Nov 22, 2022

slava77 commented Nov 4, 2022 •

edited

Loading

smuzaffar commented Nov 22, 2022 •

edited

Loading