HLT menu development for `14_1_X` (2/N) #46376

mmusich · 2024-10-14T14:32:12Z

backport of #46375

PR description:

Development of HLT menus for CMSSW_14_1_X

This is the PR for the 14_1_X branch which is the release cycle currently used for Run-3 HLT-menu development.

These updates correspond to the changes accumulated in the 14_1_X release cycle for the "PRef V1.0" menu: first menu for the 2024 pp reference data-taking period.

This update includes the integration of the following changes:

migration of the combined HLT table and related subtables to CMSSW_14_1_1

In addition we include the integration of the following JIRA tickets:

CMSHLT-3325 [HIN] Forward/UPC Triggers request for pp Reference Run 2024
CMSHLT-3338 [HIN] Muon Triggers for PbPb Run 2024
CMSHLT-3340 [HIN] Jet Triggers for pp Reference Run 2024
CMSHLT-3343 [HIN] E/gamma Triggers for pp Reference Run 2024
CMSHLT-3345 [HIN] Remove PPRefExotica PD/stream and re-assign paths to PPRefSingleMuon
CMSHLT-3346 [HIN] Justify Commissioning{ZDC,RawPrime} PDs in the PRef menu
CMSHLT-3348 [HIN] Add two seeds to hltL1sUPCHFafterglowCombined
CMSHLT-3350 [HIN] Forward/UPC Triggers request for PbPb Run 2024
CMSHLT-3356 [HIN] E/gamma Triggers for PbPb Run 2024
CMSHLT-3357 [TSG] Release of special menus V1.6 (for PRef, HIon runs)
CMSHLT-3363 [HIN,TSG] Sync offline menu with the 2023 PRef online menu

In this PR we also have the following commits:

9be857b update of L1T HIon menu in TSG GTs to L1Menu_CollisionsHeavyIons2024_v1_0_5_xml
9b44c34 Finalize integration of the following tickets
- CMSHLT-3310 [TSG] update input files for TSG tests to more modern datasets (2024)
0aaf2a4 Update hltPathOwners.json including paths for menus PRef 2024 V1.0 and HIon 2024 V1.0
2beed13 Bumped version of Paths for HLT PRef-2024 V1.0

PR validation:

TSG Integration tests.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

backport of #46375 to the 2024 PRef and HIon production cycle (CMSSW_14_1_X)

These updates correspond to the changes accumulated in the 14_1_X release cycle for the "PRef V1.0" menu: first menu for the 2024 pp reference data-taking period. This update includes the integration of the following changes: - migration of the combined HLT table and related subtables to CMSSW_14_1_1 In addition we include the integration of the following JIRA tickets: - [CMSHLT-3325](https://its.cern.ch/jira/browse/CMSHLT-3325) [HIN] Forward/UPC Triggers request for pp Reference Run 2024 - [CMSHLT-3338](https://its.cern.ch/jira/browse/CMSHLT-3338) [HIN] Muon Triggers for PbPb Run 2024 - [CMSHLT-3340](https://its.cern.ch/jira/browse/CMSHLT-3340) [HIN] Jet Triggers for pp Reference Run 2024 - [CMSHLT-3343](https://its.cern.ch/jira/browse/CMSHLT-3343) [HIN] E/gamma Triggers for pp Reference Run 2024 - [CMSHLT-3345](https://its.cern.ch/jira/browse/CMSHLT-3345) [HIN] Remove PPRefExotica PD/stream and re-assign paths to PPRefSingleMuon - [CMSHLT-3346](https://its.cern.ch/jira/browse/CMSHLT-3346) [HIN] Justify Commissioning{ZDC,RawPrime} PDs in the PRef menu - [CMSHLT-3348](https://its.cern.ch/jira/browse/CMSHLT-3348) [HIN] Add two seeds to hltL1sUPCHFafterglowCombined - [CMSHLT-3350](https://its.cern.ch/jira/browse/CMSHLT-3350) [HIN] Forward/UPC Triggers request for PbPb Run 2024 - [CMSHLT-3356](https://its.cern.ch/jira/browse/CMSHLT-3356) [HIN] E/gamma Triggers for PbPb Run 2024 - [CMSHLT-3357](https://its.cern.ch/jira/browse/CMSHLT-3357) [TSG] Release of special menus V1.6 (for PRef, HIon runs) - [CMSHLT-3363](https://its.cern.ch/jira/browse/CMSHLT-3363) [HIN,TSG] Sync offline menu with the 2023 PRef online menu

- [CMSHLT-3310](https://its.cern.ch/jira/browse/CMSHLT-3310) [TSG] update input files for TSG tests to more modern datasets (2024)

…v1_0_5_xml

…d HIon V1.0

…_0/HLT/V190)

cmsbuild · 2024-10-14T14:32:38Z

A new Pull Request was created by @mmusich for CMSSW_14_1_X.

It involves the following packages:

Configuration/HLT (hlt)
HLTrigger/Configuration (hlt)

@Martin-Grunewald, @cmsbuild, @mmusich can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @missirol, @silviodonato this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Backported from HLT menu development for 14_1_X (2/N) [14_2_X] #46375

cmsbuild · 2024-10-14T14:32:39Z

cms-bot internal usage

mmusich · 2024-10-14T14:34:08Z

type hlt-integration

mmusich · 2024-10-14T14:37:40Z

enable gpu

mmusich · 2024-10-14T18:36:17Z

@cmsbuild, please test

cmsbuild · 2024-10-14T21:39:39Z

+1

Size: This PR adds an extra 1304KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-983d4f/42175/summary.html
COMMIT: 2beed13
CMSSW: CMSSW_14_1_X_2024-10-14-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46376/42175/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3317891
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3317865
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.728 KiB( 43 files compared)
DQMHistoSizes: changed ( 12834.0,... ): 0.104 KiB HLT/Filters
Checked 191 log files, 161 edm output root files, 44 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 53015
DQMHistoTests: Total failures: 807
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 52208
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 30 edm output root files, 7 DQM output files
TriggerResults: no differences found

mmusich · 2024-10-15T06:46:28Z

+hlt

changes in the HLTFilters DQM folders are due the path versioning (2beed13).

cmsbuild · 2024-10-15T06:46:52Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mmusich · 2024-10-15T13:50:34Z

message to @cms-sw/orp-l2, this PR needs to be merged together with #46389

mmusich · 2024-10-15T13:58:13Z

test parameters:

pull_request = Update a few MC Gts in autoCond to the latest version available on 14/10/2024 - CMSSW_14_1_X #46389
workflows = 160, 160.02, 160.03, 160.1, 160.2, 160.3, 160.4, 160.501, 180.0, 180.1

mmusich · 2024-10-15T13:58:25Z

@cmsbuild, please test

mmusich · 2024-10-15T14:03:22Z

@cmsbuild, please abort

mmusich · 2024-10-15T14:03:57Z

@cmsbuild, please test

cmsbuild · 2024-10-15T18:11:57Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-983d4f/42204/summary.html
COMMIT: 2beed13
CMSSW: CMSSW_14_1_X_2024-10-15-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46376/42204/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

180.0

----- Begin Fatal Exception 15-Oct-2024 20:01:45 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

160.03

----- Begin Fatal Exception 15-Oct-2024 20:01:47 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

160.501

----- Begin Fatal Exception 15-Oct-2024 20:01:51 CEST-----------------------
An exception of category 'FailModule' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLT_HIL2SingleMu0_Centrality40to100_v1'
   [2] Calling method for module HLTL1TSeed/'hltL1sSingleMu0Centrality40to100BptxAND'
Exception Message:

Algorithm L1_SingleMu0_Centrality_40_100_BptxAND, requested as seed by a HLT path, cannot be matched to a L1 algo name in any GlobalObjectMap
Please check if algorithm L1_SingleMu0_Centrality_40_100_BptxAND is present in the L1 menu

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

GPU Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 246 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 53015
DQMHistoTests: Total failures: 26144
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 26871
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 30 edm output root files, 7 DQM output files
TriggerResults: found differences in 6 / 6 workflows

mmusich · 2024-10-15T18:34:31Z

Ah, now I get it... all the failing workflows are running the 2023 GlobalTag (which wasn't updated in terms of L1T menu). Assuming we don't want to update them to be 2024-like, we should run the fake HLT menu instead. @mandrenguyen any preference?

mandrenguyen · 2024-10-15T18:43:28Z

Fake would be fine

mmusich · 2024-10-15T19:40:01Z

Fake would be fine

see #46399

mmusich · 2024-10-15T19:40:38Z

test parameters:

pull_request = [14.1.X] Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46399
workflows = 160, 160.02, 160.03, 160.1, 160.2, 160.3, 160.4, 160.501, 180.0, 180.1

mmusich · 2024-10-15T19:40:45Z

@cmsbuild, please test

cmsbuild · 2024-10-15T23:22:51Z

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-983d4f/42217/summary.html
COMMIT: 2beed13
CMSSW: CMSSW_14_1_X_2024-10-15-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46376/42217/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 728 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 517 differences found in the comparisons
DQMHistoTests: Total files compared: 52
DQMHistoTests: Total histograms compared: 4147551
DQMHistoTests: Total failures: 3911
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4143620
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.728 KiB( 51 files compared)
DQMHistoSizes: changed ( 12834.0,... ): 0.104 KiB HLT/Filters
Checked 238 log files, 205 edm output root files, 52 DQM output files
TriggerResults: found differences in 8 / 50 workflows

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 53015
DQMHistoTests: Total failures: 30
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 52985
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 30 edm output root files, 7 DQM output files
TriggerResults: no differences found

mmusich · 2024-10-17T14:10:22Z

@cms-sw/orp-l2 we need to get going with this PR as there are more tickets to integrate. Please merge at earliest convenience.
About possible IB failures triggered by the update, they will be cured by #46399 - but they are not caused by this particular update here, but from a lack of organization in the HIon relvals structure that can be figured out at a later time.
Thanks your for the collaboration.

mandrenguyen · 2024-10-18T06:32:00Z

+1

mmusich added 5 commits October 14, 2024 14:23

Finalize integration of the following tickets

9b44c34

- [CMSHLT-3310](https://its.cern.ch/jira/browse/CMSHLT-3310) [TSG] update input files for TSG tests to more modern datasets (2024)

update of L1T HIon menu in TSG GTs to L1Menu_CollisionsHeavyIons2024_…

9be857b

…v1_0_5_xml

Update hltPathOwners.json including paths for menus PRef 2024 V1.0 an…

0aaf2a4

…d HIon V1.0

Bumped version of Paths for HLT PRef-2024 V1.0 (base: /dev/CMSSW_14_0…

2beed13

…_0/HLT/V190)

cmsbuild added this to the CMSSW_14_1_X milestone Oct 14, 2024

cmsbuild added hlt-pending pending-signatures tests-pending orp-pending backport labels Oct 14, 2024

cmsbuild added the hlt-integration label Oct 14, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 14, 2024

cmsbuild added hlt-approved fully-signed and removed hlt-pending pending-signatures labels Oct 15, 2024

cmsbuild added backport-ok and removed backport labels Oct 15, 2024

cmsbuild added tests-started and removed tests-approved labels Oct 15, 2024

cmsbuild added the requires-external label Oct 15, 2024

cmsbuild added tests-rejected and removed tests-started labels Oct 15, 2024

mmusich mentioned this pull request Oct 15, 2024

[14.1.X] Update 2023 and 2024 Heavy Ion relval workflows for 2024 HLT menu #46399

Merged

cmsbuild added tests-started and removed tests-rejected labels Oct 15, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 15, 2024

cmsbuild added orp-approved and removed orp-pending labels Oct 18, 2024

cmsbuild merged commit 7c90601 into cms-sw:CMSSW_14_1_X Oct 18, 2024
13 checks passed

mmusich deleted the HLTdevelopment141X branch October 18, 2024 06:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HLT menu development for `14_1_X` (2/N) #46376

HLT menu development for `14_1_X` (2/N) #46376

mmusich commented Oct 14, 2024

cmsbuild commented Oct 14, 2024 •

edited

Loading

cmsbuild commented Oct 14, 2024 •

edited

Loading

mmusich commented Oct 14, 2024

mmusich commented Oct 14, 2024

mmusich commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024 •

edited

Loading

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 15, 2024 •

edited

Loading

mandrenguyen commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 17, 2024

mandrenguyen commented Oct 18, 2024

HLT menu development for 14_1_X (2/N) #46376

HLT menu development for 14_1_X (2/N) #46376

Conversation

mmusich commented Oct 14, 2024

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Oct 14, 2024 • edited Loading

cmsbuild commented Oct 14, 2024 • edited Loading

mmusich commented Oct 14, 2024

mmusich commented Oct 14, 2024

mmusich commented Oct 14, 2024

cmsbuild commented Oct 14, 2024

Comparison Summary

GPU Comparison Summary

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024 • edited Loading

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

RelVals

GPU Comparison Summary

mmusich commented Oct 15, 2024 • edited Loading

mandrenguyen commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

mmusich commented Oct 15, 2024

cmsbuild commented Oct 15, 2024

Comparison Summary

GPU Comparison Summary

mmusich commented Oct 17, 2024

mandrenguyen commented Oct 18, 2024

HLT menu development for `14_1_X` (2/N) #46376

HLT menu development for `14_1_X` (2/N) #46376

cmsbuild commented Oct 14, 2024 •

edited

Loading

cmsbuild commented Oct 14, 2024 •

edited

Loading

mmusich commented Oct 15, 2024 •

edited

Loading

mmusich commented Oct 15, 2024 •

edited

Loading