[EGM HLT@Phase2] Phase out caloTowers #40525

swagata87 · 2023-01-14T05:50:18Z

PR description:

With this PR we propose to discontinue the usage of caloTowers in Phase2 E/gamma HLT.
This is done in a similar way as it was done for Run3 EGM HLT as described in [1], [2], [3], [4].

[1] https://its.cern.ch/jira/browse/CMSHLT-2286
[2] https://indico.cern.ch/event/1116957/contributions/4716661/attachments/2383419/4072743/caloTowerEGMHLT.pdf
[3] #35049
[4] #36157

PR validation:

RecHit-based H/E and CaloTower-based H/E are very close. This plot is made from CMSSW_12_4_8, using ZprimeToEE sample (PhaseIISpring22, PU200). The objects entering the plot are all the HLT-level superclusters with pT>10 GeV, in barrel.

Tagging HLT upgrade convenors @beaucero and @SohamBhattacharya

Backport of this PR is not necessary.

cmsbuild · 2023-01-14T05:56:37Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40525/33732

This PR adds an extra 36KB to repository

cmsbuild · 2023-01-14T05:57:04Z

A new Pull Request was created by @swagata87 (Swagata Mukherjee) for master.

It involves the following packages:

HLTrigger/Configuration (hlt)

@cmsbuild, @missirol, @Martin-Grunewald can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @silviodonato this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

missirol

I'm a bit confused.

Doesn't the change to hltFixedGridRhoFastjetAllCaloForMuons also affect the muon reconstruction?

If that is intended, and the same module is used by both EGM and MUO, the postfix "ForMuons" could be removed for clarity.

Several EGM modules (e.g. hltDiEG2312IsoHEL1SeededFilter) still reference hltFixedGridRhoFastjetAllCaloForEGamma (which doesn't exist); I guess it does not matter because doRhoCorrection = False, but this should be replaced with an empty InputTag (like in hltDiEG2312IsoClusterShapeL1SeededFilter), or the name of an appropriate (existing) collection (e.g. hltFixedGridRhoFastjetAllCalo(ForMuons)).

The files hltTowerMakerForAllForEgamma_cfi and hltRegionalTowerForEgamma_cfi should be removed if not used (unless there is a good reason to keep them).

missirol · 2023-01-14T13:19:34Z

HLTrigger/Configuration/python/HLT_75e33/modules/hltFixedGridRhoFastjetAllCaloForMuons_cfi.py

+    eThresHB = cms.vdouble( 0.1, 0.2, 0.3, 0.3 ),
+    eThresHE = cms.vdouble( 0.1, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2 )


These look like the Run-3 thresholds. Are they the same for Phase 2 ?

Let's double check this with @cms-sw/hcal-dpg-l2 and @cms-sw/pf-l2
Is the following still valid for phase2? Or do we have new PF recHit threshold for HB for phase2?

cmssw/RecoParticleFlow/PFClusterProducer/python/particleFlowRecHitHBHE_cfi.py

Line 5 in 33ef057

_thresholdsHBphase1 = cms.vdouble(0.1, 0.2, 0.3, 0.3)

I thought this depends on the 'aging' scenario, e.g.

cmssw/SLHCUpgradeSimulations/Configuration/python/aging.py

Line 96 in af4b7ab

"rec": [0.8, 1.2, 1.2, 1.2],

, and it would match what is used in the modules below [*], where there are already some differences.

Indeed, it would be good to clarify this.

[*]

cmssw/HLTrigger/Configuration/python/HLT_75e33/modules/particleFlowRecHitHBHE_cfi.py

Line 16 in af4b7ab

threshold = cms.vdouble(0.8, 1.2, 1.2, 1.2)

cmssw/HLTrigger/Configuration/python/HLT_75e33/modules/particleFlowClusterHBHE_cfi.py

Line 13 in af4b7ab

gatheringThreshold = cms.vdouble(0.8, 1.2, 1.2, 1.2),

cmssw/HLTrigger/Configuration/python/HLT_75e33/modules/hltParticleFlowRecHitHBHEForEgamma_cfi.py

Line 25 in af4b7ab

threshold = cms.vdouble(0.8, 0.8, 0.8, 0.8)

cmssw/HLTrigger/Configuration/python/HLT_75e33/modules/hltParticleFlowClusterHBHEForEgamma_cfi.py

Line 13 in af4b7ab

gatheringThreshold = cms.vdouble(0.8, 0.8, 0.8, 0.8),

I was about to point to usual Phase2 aging customization
https://github.com/cms-sw/cmssw/blob/master/SLHCUpgradeSimulations/Configuration/python/aging.py#245
With PFlowClusters thresholds/seeds involved.
But just noticed that Marino has already commented on it just several minutes ago (!)

Typical Phase2 use case is 1000/fb in this list:
https://github.com/cms-sw/cmssw/blob/master/SLHCUpgradeSimulations/Configuration/python/aging.py#L89-#L127

swagata87 · 2023-01-15T12:43:33Z

Doesn't the change to hltFixedGridRhoFastjetAllCaloForMuons also affect the muon reconstruction?

Indeed it does. But for now RhoCorrection is disabled in muon HLT also, so no difference will show up in tests.

If that is intended, and the same module is used by both EGM and MUO, the postfix "ForMuons" could be removed for clarity.

I have now kept hltFixedGridRhoFastjetAllCaloForMuons as it is and created a new file hltFixedGridRhoFastjetAllCaloForEGamma. This way, later on MUO POG can configure the hltFixedGridRhoFastjetAllCaloForMuons differently than egamma if they need to.

Several EGM modules (e.g. hltDiEG2312IsoHEL1SeededFilter) still reference hltFixedGridRhoFastjetAllCaloForEGamma (which doesn't exist); I guess it does not matter because doRhoCorrection = False, but this should be replaced with an empty InputTag (like in hltDiEG2312IsoClusterShapeL1SeededFilter), or the name of an appropriate (existing) collection (e.g. hltFixedGridRhoFastjetAllCalo(ForMuons)).

this issue is now solved as I have now introduced a new module named hltFixedGridRhoFastjetAllCaloForEGamma

The files hltTowerMakerForAllForEgamma_cfi and hltRegionalTowerForEgamma_cfi should be removed if not used (unless there is a good reason to keep them).

these 2 files are now removed

The latest changes are tested with runTheMatrix.py -l 20834.76

cmsbuild · 2023-01-15T12:52:01Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40525/33733

This PR adds an extra 36KB to repository

cmsbuild · 2023-01-15T12:52:25Z

Pull request #40525 was updated. @cmsbuild, @missirol, @Martin-Grunewald can you please check and sign again.

cmsbuild · 2023-01-16T08:35:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40525/33736

This PR adds an extra 40KB to repository

cmsbuild · 2023-01-16T08:35:51Z

Pull request #40525 was updated. @cmsbuild, @missirol, @Martin-Grunewald can you please check and sign again.

swagata87 · 2023-01-16T08:44:34Z

thank you Marino and Salavat for pointing out the correct threshold values. I updated it now.
Due to higher values of thresholds the distributions of HCAL related quantities are now slightly shifted to lower values as expected. I don't think this is a problem, and if needed the cut values can be re-optimised at a later time.

In the following plots, Black is base, and Green is [base+PR]
H/E

HCAL PF cluster Isolation (this is unrelated to caloTower->RecHit change. This variable changed as I updated HCAL thresholds in hltParticleFlowRecHitHBHEForEgamma_cfi.py and hltParticleFlowClusterHBHEForEgamma_cfi.py.

I take note of the fact that there are many other differences between modules/particleFlowClusterHBHE_cfi.py and modules/hltParticleFlowClusterHBHEForEgamma_cfi.py. It looks like we could align these 2 modules, and the egamma version of the module needs some update. But it might need some more time to figure this out. If it's fine, it can be done in a followup PR.

missirol · 2023-01-16T09:02:07Z

please test

If it's fine, it can be done in a followup PR.

That'd be okay with me.

cmsbuild · 2023-01-16T11:23:20Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cdd063/30000/summary.html
COMMIT: fda2b10
CMSSW: CMSSW_13_0_X_2023-01-15-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40525/30000/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555538
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555510
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

missirol · 2023-01-16T12:54:16Z

+hlt

cmsbuild · 2023-01-16T12:54:37Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-01-16T15:55:48Z

+1

phase out calotowers

1341348

cmsbuild added this to the CMSSW_13_0_X milestone Jan 14, 2023

cmsbuild added code-checks-pending hlt-pending orp-pending pending-signatures tests-pending labels Jan 14, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 14, 2023

missirol reviewed Jan 14, 2023

View reviewed changes

addressed Marino's comments

0171145

cmsbuild added code-checks-pending and removed code-checks-approved labels Jan 15, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 15, 2023

use correct thresholds for HCAL

fda2b10

cmsbuild added code-checks-pending and removed code-checks-approved labels Jan 16, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 16, 2023

cmsbuild removed the tests-pending label Jan 16, 2023

cmsbuild added the tests-started label Jan 16, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 16, 2023

cmsbuild added fully-signed hlt-approved and removed hlt-pending pending-signatures labels Jan 16, 2023

cmsbuild added orp-approved and removed orp-pending labels Jan 16, 2023

cmsbuild merged commit 9c9510b into cms-sw:master Jan 16, 2023

cmsbuild mentioned this pull request Jan 16, 2023

Adding new GCP trends tool and commenting unnecessary print out in Trend #40336

Merged

swagata87 mentioned this pull request Mar 14, 2023

Adapt latest PF settings in Phase2 Egamma HLT #41051

Merged

wonpoint4 mentioned this pull request May 23, 2023

[MuonHLT@Phase2] Adapt PF Settings in Phase2 MuonHLT #41749

Merged

swagata87 mentioned this pull request Jan 3, 2024

Calotower phase-out from PF code #43612

Merged

swagata87 mentioned this pull request May 20, 2024

RecHit based Ecal/Hcal muon isolation #44797

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EGM HLT@Phase2] Phase out caloTowers #40525

[EGM HLT@Phase2] Phase out caloTowers #40525

swagata87 commented Jan 14, 2023

cmsbuild commented Jan 14, 2023

cmsbuild commented Jan 14, 2023

missirol left a comment

missirol Jan 14, 2023

swagata87 Jan 15, 2023

missirol Jan 15, 2023

abdoulline Jan 15, 2023 •

edited

Loading

swagata87 commented Jan 15, 2023

cmsbuild commented Jan 15, 2023

cmsbuild commented Jan 15, 2023

cmsbuild commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

swagata87 commented Jan 16, 2023

missirol commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

missirol commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

perrotta commented Jan 16, 2023

		eThresHB = cms.vdouble( 0.1, 0.2, 0.3, 0.3 ),
		eThresHE = cms.vdouble( 0.1, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2 )

[EGM HLT@Phase2] Phase out caloTowers #40525

[EGM HLT@Phase2] Phase out caloTowers #40525

Conversation

swagata87 commented Jan 14, 2023

PR description:

PR validation:

cmsbuild commented Jan 14, 2023

cmsbuild commented Jan 14, 2023

missirol left a comment

Choose a reason for hiding this comment

missirol Jan 14, 2023

Choose a reason for hiding this comment

swagata87 Jan 15, 2023

Choose a reason for hiding this comment

missirol Jan 15, 2023

Choose a reason for hiding this comment

abdoulline Jan 15, 2023 • edited Loading

Choose a reason for hiding this comment

swagata87 commented Jan 15, 2023

cmsbuild commented Jan 15, 2023

cmsbuild commented Jan 15, 2023

cmsbuild commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

swagata87 commented Jan 16, 2023

missirol commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

Comparison Summary

missirol commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

perrotta commented Jan 16, 2023

abdoulline Jan 15, 2023 •

edited

Loading