Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

makortel · 2025-01-07T00:02:33Z

PR description:

Before #43969 a common pattern to copy data from configuration to all devices was to use the EventSetup system. While that works, it adds more pieces to be managed and disconnects the configuration parameters of a module (in addition to slight abuse of EventSetup system as "a job" is not a real IOV). This PR proposes to replace such Alpaka ESProducers with the MoveToDeviceCache introduced in #43969.

Resolves cms-sw/framework-team#1129

PR validation:

Code compiles, matrix workflows 12834.{402,412,422} and 145.604 run with and without an NVIDIA GPU.

cmsbuild · 2025-01-07T00:03:03Z

cms-bot internal usage

cmsbuild · 2025-01-07T00:05:46Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43186

There are other open Pull requests which might conflict with changes you have proposed:
- File HLTrigger/Configuration/python/customizeHLTforCMSSW.py modified in PR(s): add fillDescriptions to a bunch of plugins in the EventFilter and Reco areas used at HLT (1/N) #47017
- File RecoLocalCalo/EcalRecProducers/plugins/alpaka/TimeComputationKernels.h modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoLocalCalo/EcalRecProducers/python/ecalMultiFitUncalibRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/EcalRecProducers/python/ecalRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/HcalRecProducers/plugins/alpaka/Mahi.dev.cc modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducer.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.dev.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632, Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.h modified in PR(s): Extend CMSSW to a distributed application over MPI #32632

cmsbuild · 2025-01-07T00:06:16Z

A new Pull Request was created by @makortel for master.

It involves the following packages:

CondFormats/DataRecord (db, alca)
CondFormats/EcalObjects (db, alca)
CondFormats/HcalObjects (db, alca)
HLTrigger/Configuration (hlt)
RecoLocalCalo/EcalRecProducers (reconstruction)
RecoLocalCalo/HcalRecProducers (reconstruction)
RecoParticleFlow/PFClusterProducer (reconstruction)

@Martin-Grunewald, @atpathak, @cmsbuild, @consuegs, @francescobrivio, @jfernan2, @mandrenguyen, @mmusich, @perrotta can you please review it and eventually sign? Thanks.
@JanChyczynski, @Martin-Grunewald, @PonIlya, @ReyerBand, @abdoulline, @apsallid, @argiro, @bsunanda, @denizsun, @felicepantaleo, @hatakeyamak, @lgray, @mariadalfonso, @missirol, @mmarionncern, @mmusich, @rchatter, @rovere, @rsreds, @salimcerci, @sameasy, @seemasharmafnal, @silviodonato, @thomreis, @tocheng, @wang0jin, @youyingli, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

makortel · 2025-01-07T00:09:12Z

@cms-sw/heterogeneous-l2

makortel · 2025-01-07T00:10:22Z

enable gpu

cmsbuild · 2025-01-07T00:11:55Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43187

There are other open Pull requests which might conflict with changes you have proposed:
- File HLTrigger/Configuration/python/customizeHLTforCMSSW.py modified in PR(s): add fillDescriptions to a bunch of plugins in the EventFilter and Reco areas used at HLT (1/N) #47017
- File RecoLocalCalo/EcalRecProducers/plugins/alpaka/TimeComputationKernels.h modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoLocalCalo/EcalRecProducers/python/ecalMultiFitUncalibRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/EcalRecProducers/python/ecalRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/HcalRecProducers/plugins/alpaka/Mahi.dev.cc modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducer.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.dev.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632, Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.h modified in PR(s): Extend CMSSW to a distributed application over MPI #32632

makortel · 2025-01-07T00:12:21Z

@cmsbuild, please test

cmsbuild · 2025-01-07T00:22:51Z

Pull request #47047 was updated. @Martin-Grunewald, @atpathak, @consuegs, @francescobrivio, @jfernan2, @mandrenguyen, @mmusich, @perrotta can you please check and sign again.

cmsbuild · 2025-01-07T02:30:54Z

+1

Size: This PR adds an extra 140KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-767973/43645/summary.html
COMMIT: bac91f6
CMSSW: CMSSW_15_0_X_2025-01-06-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47047/43645/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3818730
DQMHistoTests: Total failures: 434
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3818276
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 53071
DQMHistoTests: Total failures: 874
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 52197
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 30 edm output root files, 7 DQM output files
TriggerResults: no differences found

jfernan2 · 2025-01-07T09:52:31Z

+1

cmsbuild · 2025-01-13T16:57:50Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47047/43276

There are other open Pull requests which might conflict with changes you have proposed:
- File HLTrigger/Configuration/python/customizeHLTforCMSSW.py modified in PR(s): CA Extension to strips #47090
- File RecoLocalCalo/EcalRecProducers/plugins/alpaka/TimeComputationKernels.h modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoLocalCalo/EcalRecProducers/python/ecalMultiFitUncalibRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/EcalRecProducers/python/ecalRecHit_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
- File RecoLocalCalo/HcalRecProducers/plugins/alpaka/Mahi.dev.cc modified in PR(s): Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducer.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.dev.cc modified in PR(s): Extend CMSSW to a distributed application over MPI #32632, Replace TAcc with explicit accelerator type #46916
- File RecoParticleFlow/PFClusterProducer/plugins/alpaka/PFClusterSoAProducerKernel.h modified in PR(s): Extend CMSSW to a distributed application over MPI #32632

cmsbuild · 2025-01-13T16:58:13Z

Pull request #47047 was updated. @cmsbuild, @fwyzard, @jfernan2, @makortel, @mandrenguyen can you please check and sign again.

makortel · 2025-01-13T16:59:52Z

@cmsbuild, please test

fwyzard · 2025-01-13T17:26:44Z

+heterogeneous
(trusting that the only change is the rebase)

makortel · 2025-01-13T18:07:56Z

(trusting that the only change is the rebase)

That is correct.

cmsbuild · 2025-01-13T19:08:53Z

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-767973/43742/summary.html
COMMIT: c43fa27
CMSSW: CMSSW_15_0_X_2025-01-13-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47047/43742/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3818780
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3818754
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 53071
DQMHistoTests: Total failures: 872
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 52199
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 24 log files, 30 edm output root files, 7 DQM output files
TriggerResults: no differences found

jfernan2 · 2025-01-14T15:41:26Z

+1

cmsbuild · 2025-01-14T15:41:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2025-01-14T15:47:34Z

+1

mmusich · 2025-02-18T15:31:18Z

HLTrigger/Configuration/python/customizeHLTforCMSSW.py

+        delattr(process, esProducer.label())
+
+    for prod in producers_by_type(process, "HBHERecHitProducerPortable@alpaka", "alpaka_serial_sync::HBHERecHitProducerPortable"):
+        pulseOffsetLabel = prod.mahiPulseOffSets.getModuleLabel()


I realize only now, that this could have been written better. If one tries to run this customization function on a menu that has already the customization re-absorbed (e.g. /dev/CMSSW_15_0_0/GRun/V4) one incurs in runtime errors such as: 'EDProducer' object has no attribute 'mahiPulseOffSets'.

@Sam-Harper FYI

If one tries to run this customization function on a menu that has already the customization re-absorbed

Is this situation something these customize functions should generally work? I'm sorry I didn't think about that (I though the customize function would be discarded after the menu itself has been updated).

Is this situation something these customize functions should generally work?

when we are transitioning between release cycles, yes, it happens commonly.

This has been addressed at #47393 and backported at #47395

cmsbuild added this to the CMSSW_15_0_X milestone Jan 7, 2025

cmsbuild added reconstruction-pending hlt-pending alca-pending db-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jan 7, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 7, 2025

makortel force-pushed the useCachesInsteadOfESProducers branch from be750b4 to bac91f6 Compare January 7, 2025 00:08

cmsbuild added code-checks-pending and removed code-checks-approved labels Jan 7, 2025

cmsbuild added tests-started code-checks-approved and removed tests-pending code-checks-pending labels Jan 7, 2025

cmsbuild added tests-approved and removed tests-started labels Jan 7, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 13, 2025

cmsbuild added tests-started and removed tests-pending labels Jan 13, 2025

cmsbuild added heterogeneous-approved and removed heterogeneous-pending labels Jan 13, 2025

cmsbuild added tests-approved and removed tests-started labels Jan 13, 2025

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Jan 14, 2025

cmsbuild added orp-approved and removed orp-pending labels Jan 14, 2025

cmsbuild merged commit dcded52 into cms-sw:master Jan 14, 2025
14 checks passed

makortel deleted the useCachesInsteadOfESProducers branch January 14, 2025 17:04

cmsbuild mentioned this pull request Jan 14, 2025

revert #46026 (L1-uGT emulator) #47088

Merged

missirol mentioned this pull request Jan 17, 2025

potential speedup in HLT: AXOL1TLCondition::evaluateCondition consider caching the model? #46740

Open

mmusich reviewed Feb 18, 2025

View reviewed changes

This was referenced Feb 18, 2025

fix customizeHLTfor47047 to work also on menus already customized #47393

Merged

[15.0.X] fix customizeHLTfor47047 to work also on menus already customized #47395

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

makortel commented Jan 7, 2025

cmsbuild commented Jan 7, 2025 •

edited

Loading

cmsbuild commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

makortel commented Jan 7, 2025

makortel commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

makortel commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

jfernan2 commented Jan 7, 2025

cmsbuild commented Jan 13, 2025

cmsbuild commented Jan 13, 2025

makortel commented Jan 13, 2025

fwyzard commented Jan 13, 2025

makortel commented Jan 13, 2025

cmsbuild commented Jan 13, 2025

jfernan2 commented Jan 14, 2025

cmsbuild commented Jan 14, 2025

mandrenguyen commented Jan 14, 2025

mmusich Feb 18, 2025 •

edited

Loading

makortel Feb 18, 2025

mmusich Feb 18, 2025 •

edited

Loading

mmusich Feb 19, 2025

Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

Replace configuration-copying Alpaka ESProducers with MoveToDeviceCache #47047

Conversation

makortel commented Jan 7, 2025

PR description:

PR validation:

cmsbuild commented Jan 7, 2025 • edited Loading

cmsbuild commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

makortel commented Jan 7, 2025

makortel commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

makortel commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

cmsbuild commented Jan 7, 2025

Comparison Summary

GPU Comparison Summary

jfernan2 commented Jan 7, 2025

cmsbuild commented Jan 13, 2025

cmsbuild commented Jan 13, 2025

makortel commented Jan 13, 2025

fwyzard commented Jan 13, 2025

makortel commented Jan 13, 2025

cmsbuild commented Jan 13, 2025

Comparison Summary

GPU Comparison Summary

jfernan2 commented Jan 14, 2025

cmsbuild commented Jan 14, 2025

mandrenguyen commented Jan 14, 2025

mmusich Feb 18, 2025 • edited Loading

Choose a reason for hiding this comment

makortel Feb 18, 2025

Choose a reason for hiding this comment

mmusich Feb 18, 2025 • edited Loading

Choose a reason for hiding this comment

mmusich Feb 19, 2025

Choose a reason for hiding this comment

cmsbuild commented Jan 7, 2025 •

edited

Loading

mmusich Feb 18, 2025 •

edited

Loading

mmusich Feb 18, 2025 •

edited

Loading