Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46398

mmusich · 2024-10-15T19:23:07Z

PR description:

PR #46375 updated the HIon HLT menu such that it requires a L1T menu (L1Menu_CollisionsHeavyIons2024_v1_0_5_xml ) different from the one used in 2023 (more L1T seeds are required by trigger paths introduced in the meanwhile, see link to payload inspector [1]).
The goal of this PR is to update the 2023 HI relval workflow to use the @fake2 menu and to update the 2024 HI workflows to use the updated HIon menu (consequently the DQM and validation steps are updated to run the HLT sequences).

PR validation:

Tested successfully

runTheMatrix.py -l 160,160.02,160.03,160.1,160.2,160.3,160.4,160.501,161,161.02,161.03,161.1,161.2,161.3,161.4,180.0,180.1,181,181.1 --ibeos

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, will be backported to CMSSW_14_1_X

[1]

L1T menu comparison

cmsbuild · 2024-10-15T19:23:29Z

cms-bot internal usage

cmsbuild · 2024-10-15T19:24:47Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46398/42245

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): New Inputs for D110 And Enabling HIN Phase2 Wf #46355

cmsbuild · 2024-10-15T19:25:05Z

A new Pull Request was created by @mmusich for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)

@AdrianoDee, @Moanwar, @cmsbuild, @kskovpen, @miquork, @srimanob, @subirsarkar, @sunilUIET can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @makortel, @missirol, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2024-10-15T19:26:02Z

test parameters:

workflows = 160, 160.02, 160.03, 160.1, 160.2, 160.3, 160.4, 160.501, 161, 161.02, 161.03, 161.1, 161.2, 161.3, 161.4, 180.0, 180.1, 181, 181.1

mmusich · 2024-10-15T19:28:09Z

@cmsbuild, please test

…lidation in RECOHI2024 steps

mmusich · 2024-10-15T21:36:14Z

@cmsbuild, please test

mmusich · 2024-10-15T21:36:27Z

urgent

fixes IB relval failures

cmsbuild · 2024-10-15T21:36:47Z

Pull request #46398 was updated. @AdrianoDee, @Moanwar, @kskovpen, @miquork, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

cmsbuild · 2024-10-15T23:43:46Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 60KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6c9bcd/42224/summary.html
COMMIT: 3765a26
CMSSW: CMSSW_14_2_X_2024-10-15-1400/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46398/42224/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

161.03

----- Begin Fatal Exception 16-Oct-2024 01:24:11 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

161.0

----- Begin Fatal Exception 16-Oct-2024 01:24:35 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

161.3

----- Begin Fatal Exception 16-Oct-2024 01:24:42 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

mmusich · 2024-10-16T05:30:59Z

CMSSW: CMSSW_14_2_X_2024-10-15-1400

Sigh, an IB without #46379 was picked by the bot. Let's retest

mmusich · 2024-10-16T05:31:09Z

@cmsbuild, please test

cmsbuild · 2024-10-16T09:18:46Z

+1

Size: This PR adds an extra 60KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6c9bcd/42225/summary.html
COMMIT: 3765a26
CMSSW: CMSSW_14_2_X_2024-10-15-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46398/42225/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
160.02 step 2
160.03 step 2
160.0 step 2
160.1 step 2
160.2 step 2
160.3 step 2
160.4 step 2
160.501 step 2
180.0 step 2
180.1 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially added 845 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 54 differences found in the comparisons
DQMHistoTests: Total files compared: 52
DQMHistoTests: Total histograms compared: 4165589
DQMHistoTests: Total failures: 465
DQMHistoTests: Total nulls: 4
DQMHistoTests: Total successes: 4165100
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 809242.4160000001 KiB( 51 files compared)
DQMHistoSizes: changed ( 161.02,... ): 17547.388 KiB HLT/Tracking
DQMHistoSizes: changed ( 161.02,... ): 12810.146 KiB HLT/EGM
DQMHistoSizes: changed ( 161.02,... ): 10390.469 KiB HLT/SiStrip
DQMHistoSizes: changed ( 161.02,... ): 10237.354 KiB HLT/HCAL
DQMHistoSizes: changed ( 161.02,... ): 9249.430 KiB HLT/Vertexing
DQMHistoSizes: changed ( 161.02,... ): 9066.871 KiB HLT/Pixel
DQMHistoSizes: changed ( 161.02,... ): 8601.007 KiB HLT/Muon
DQMHistoSizes: changed ( 161.02,... ): 8296.829 KiB HLT/BTV
DQMHistoSizes: changed ( 161.02,... ): 5317.474 KiB HLT/SUSYBSM
DQMHistoSizes: changed ( 161.02,... ): 2757.393 KiB HLT/BPH
DQMHistoSizes: changed ( 161.02 ): ...
Checked 255 log files, 221 edm output root files, 52 DQM output files
TriggerResults: found differences in 8 / 50 workflows

mandrenguyen · 2024-10-16T09:22:33Z

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 please have a look at this and the 14_1_X backport.

Moanwar · 2024-10-16T09:28:56Z

+Upgrade

AdrianoDee · 2024-10-16T10:01:55Z

+pdmv

cmsbuild · 2024-10-16T10:02:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-10-16T10:47:40Z

+1

mmusich · 2024-10-17T07:13:19Z

Configuration/PyReleaseValidation/python/relval_steps.py

-steps['DIGIHI2024APPROXCLUSTERS']=merge([{'-s':'DIGI:pdigi_hi_nogen,L1,DIGI2RAW,HLT:@fake2'}, hiDefaults2024_approxClusters, {'--pileup':'HiMixNoPU'}, step2Upg2015Defaults])
-steps['DIGIHI2024']=merge([{'-s':'DIGI:pdigi_hi_nogen,L1,DIGI2RAW,HLT:@fake2'}, hiDefaults2024, {'--pileup':'HiMixNoPU'}, step2Upg2015Defaults])
+steps['DIGIHI2024APPROXCLUSTERS']=merge([{'-s':'DIGI:pdigi_hi_nogen,L1,DIGI2RAW,HLT:HIon'}, hiDefaults2024_approxClusters, {'--pileup':'HiMixNoPU'}, step2Upg2015Defaults])
+steps['DIGIHI2024']=merge([{'-s':'DIGI:pdigi_hi_nogen,L1,DIGI2RAW,HLT:HIon'}, hiDefaults2024, {'--pileup':'HiMixNoPU'}, step2Upg2015Defaults])


Looks like DIGIHI2024MIX should have been updated too log.

Followed up in #46418

use fake HLT menu for DIGIHI2023PPRECO relval step

c77ee15

cmsbuild added this to the CMSSW_14_2_X milestone Oct 15, 2024

cmsbuild added pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending labels Oct 15, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 15, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 15, 2024

mmusich changed the title ~~Update 2023 and 2024 Heavy Ion relval worflows for 2024 HLT menu~~ Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu Oct 15, 2024

This was referenced Oct 15, 2024

[14.1.X] Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46399

Merged

HLT menu development for 14_1_X (2/N) [14_2_X] #46375

Merged

Run HIon HLT menu in DIGIHI2024* steps and standardDQM and standardVa…

3765a26

…lidation in RECOHI2024 steps

mmusich force-pushed the mm_dev_2023_2024_HIon_relvals branch from 7a91d91 to 3765a26 Compare October 15, 2024 21:35

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-approved labels Oct 15, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 15, 2024

cmsbuild added tests-rejected and removed tests-started labels Oct 15, 2024

cmsbuild added tests-started and removed tests-rejected labels Oct 16, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 16, 2024

cmsbuild added upgrade-approved and removed upgrade-pending labels Oct 16, 2024

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Oct 16, 2024

cmsbuild added orp-approved and removed orp-pending labels Oct 16, 2024

cmsbuild merged commit a86af1c into cms-sw:master Oct 16, 2024
12 checks passed

mmusich deleted the mm_dev_2023_2024_HIon_relvals branch October 16, 2024 10:48

mmusich mentioned this pull request Oct 16, 2024

Add dedxLikelihood to 2023 UPC rereco #46401

Merged

mmusich commented Oct 17, 2024

View reviewed changes

mmusich mentioned this pull request Oct 17, 2024

make {DIGI,RECO}HI2024MIX steps consume the HIon HLT menu #46418

Merged

cmsbuild mentioned this pull request Oct 18, 2024

[vdt]Apply patch to avoid UBSan signed integer overflow error cms-sw/cmsdist#9470

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46398

Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46398

mmusich commented Oct 15, 2024 •

edited

Loading

cmsbuild commented Oct 15, 2024 •

edited

Loading

cmsbuild commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 15, 2024 •

edited

Loading

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 16, 2024

mmusich commented Oct 16, 2024

cmsbuild commented Oct 16, 2024

mandrenguyen commented Oct 16, 2024

Moanwar commented Oct 16, 2024

AdrianoDee commented Oct 16, 2024

cmsbuild commented Oct 16, 2024

mandrenguyen commented Oct 16, 2024

mmusich Oct 17, 2024

mmusich Oct 17, 2024

Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46398

Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46398

Conversation

mmusich commented Oct 15, 2024 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Oct 15, 2024 • edited Loading

cmsbuild commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 15, 2024 • edited Loading

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

RelVals

mmusich commented Oct 16, 2024

mmusich commented Oct 16, 2024

cmsbuild commented Oct 16, 2024

Comparison Summary

mandrenguyen commented Oct 16, 2024

Moanwar commented Oct 16, 2024

AdrianoDee commented Oct 16, 2024

cmsbuild commented Oct 16, 2024

mandrenguyen commented Oct 16, 2024

mmusich Oct 17, 2024

Choose a reason for hiding this comment

mmusich Oct 17, 2024

Choose a reason for hiding this comment

mmusich commented Oct 15, 2024 •

edited

Loading

cmsbuild commented Oct 15, 2024 •

edited

Loading

mmusich commented Oct 15, 2024 •

edited

Loading