Add Alpaka implementation of PFRecHitProducer #42807

fllor · 2023-09-18T10:28:00Z

PR description:

This pull-request is the first step towards making the ParticleFlow algorithm portable and heterogeneous using Alpaka. It adds a PFRecHitSoA data format and a producer that constructs it. The module can process HCAL and ECAL hits and runs on the CPU and CUDA backends of Alpaka. The ROCm backend compiles, but is untested.
Since calorimeter reconstruction is not available using Alpaka yet, this PR also introduces a CaloRecHitSoA format and a producer that converts calorimeter hits from the CPU modules to the SoA. This format and producer will become obsolete, when then HCAL/ECAL groups directly provide their own SoA formats.
A producer that converts to new PFRecHitSoA format back to the existing PFRecHit collection format is included. It can be used for validation and to run down-stream modules.
This PR does not alter any existing workflow in any way. The Alpaka module is provided but not executed for HLT reconstruction. For now, it can be used for testing and for down-stream development. The transition from the CPU implementation to Alpaka is intended to happen later, when an Alpaka implementation of the PFClusterProducer is also available.

PR validation:

Extensive validation of the Alpaka implementation is provided, see commit message of 3f4d123.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is not a backport.

@hatakeyamak @jsamudio

cmsbuild · 2023-09-18T10:37:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42807/36921

This PR adds an extra 204KB to repository

cmsbuild · 2023-09-18T10:38:20Z

A new Pull Request was created by @fllor for master.

It involves the following packages:

DataFormats/ParticleFlowReco (reconstruction)
RecoParticleFlow/PFRecHitProducer (****)

The following packages do not have a category, yet:

RecoParticleFlow/PFRecHitProducer
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@mmarionncern, @rchatter, @wang0jin, @rovere, @argiro, @missirol, @thomreis, @hatakeyamak, @lgray, @seemasharmafnal this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Required for CMSSW PR cms-sw/cmssw#42807

fllor · 2023-09-18T11:24:23Z

The following packages do not have a category, yet:
RecoParticleFlow/PFRecHitProducer Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

PR created: cms-sw/cms-bot/pull/2070

makortel · 2023-09-18T13:34:03Z

assign heterogeneous

cmsbuild · 2023-09-18T13:34:33Z

New categories assigned: heterogeneous

@fwyzard,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks

hatakeyamak · 2023-09-18T17:35:56Z

It may be worthwhile to mention that @filor will talk about the work that went into this PR tomorrow in the particle flow group meeting.
https://indico.cern.ch/event/1326067/
@swagata87 [I don't know Riccardo's github username.]

DataFormats/ParticleFlowReco/BuildFile.xml

DataFormats/ParticleFlowReco/src/alpaka/classes_serial.h

DataFormats/ParticleFlowReco/src/alpaka/classes_serial_def.xml

makortel · 2023-09-18T13:47:02Z

RecoParticleFlow/PFRecHitProducer/BuildFile.xml

I wonder if a new package is warranted. IIUC the current PFRecHitProducer lives in RecoParticleFlow/PFClusterProducer. On the other hand, the set of dependencies is quite different between the packages.

RecoParticleFlow/PFRecHitProducer/test/pfRecHitProducerAlpakaValidation.py

RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitProducerKernel.dev.cc

RecoParticleFlow/PFRecHitProducer/plugins/alpaka/PFRecHitTopologyESProducer.cc

RecoParticleFlow/PFRecHitProducer/test/harvestValidationPlot.py

swagata87 · 2023-09-19T07:07:36Z

FYI @cms-sw/pf-l2 @bellan

swagata87 · 2023-09-27T18:14:41Z

type pf

1) Merge files src/alpaka/classes_serial{.h,_def.xml} into src/classes{.h,_def.xml} and update BuildFile.xml 2) Split class definitions into two files, undoing commit 42fd4a7. This is possible due to PR cms-sw/cmsdist/pull/8705. Switching to this version required a small change to the Python configuration. These two changes were suggested for PR cms-sw#42807.

hatakeyamak · 2023-10-16T17:33:34Z

@fllor Just checking in where we stand.
Do you think you addressed all suggestions from @makortel et al? If so, it will be good to explicitly clarify that.

hatakeyamak · 2023-10-16T17:35:42Z

BTW, we should also adopt GT based HCAL threshold in the alpaka version too. Maybe in a follow-up PR.
@fllor @jsamudio

fwyzard · 2023-10-17T13:37:54Z

@fllor I will send you some commits to address some technical and stylish issues:

fix the dictionaries under DataFormats/ParticleFlowReco;
when the host types are in the reco namespace, move the device types to the ALPAKA_ACCELERATOR_NAMESPACE::reco namespace;
make the host types available in the ALPAKA_ACCELERATOR_NAMESPACE::reco namespace;
clean up the header guards;
update the names of structs and constants to follow the CMS coding rules (types are in Uppercase, variables and constants are in lowercase).

fwyzard · 2023-10-17T13:42:11Z

Please pick the ~~three~~ four commits from https://github.com/fwyzard/cmssw/tree/pfRecHitProducerAlpaka .

…package 1) Allocate buffers used for PFRecHitProducerKernel per-event. This allows making the producer global. 2) Change ::Grids synchronisation to ::Blocks synchronisation for GPU backend. This removes the need to distinguish between CPU and GPU backends in the kernel code. 3) Simplify code; add comments; more meaningful names; adapt CMSSW code style These two changes were suggested for PR cms-sw#42807.

cmsbuild · 2023-10-25T13:25:09Z

Pull request #42807 was updated. @mandrenguyen, @fwyzard, @jfernan2, @makortel, @cmsbuild can you please check and sign again.

fllor · 2023-10-25T13:26:19Z

I combined all the commits from the review into the original ones. I tried to put everything into a logical sequence, with different parts of the module being introduced in separate commits. Hopefully this can help to understand the code, in case someone looks at it in the future.
I think I have now addressed every suggestion from the review. Please let me know if I missed anything.

fwyzard · 2023-10-25T14:29:03Z

+heterogeneous

fwyzard · 2023-10-25T14:29:07Z

please test

hatakeyamak · 2023-10-25T16:16:52Z

Thank you. So, in terms of signatures, we still need it from reconstruction and pf?

fwyzard · 2023-10-25T18:38:23Z

Thank you. So, in terms of signatures, we still need it from reconstruction and pf?

Technically, only from @cms-sw/reconstruction-l2, while @cms-sw/pf-l2 is only "for information" and does not require a signature.

cmsbuild · 2023-10-25T19:38:55Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cb4096/35413/summary.html
COMMIT: 3f4d123
CMSSW: CMSSW_13_3_X_2023-10-24-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42807/35413/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 69 lines to the logs
Reco comparison results: 29 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3351458
DQMHistoTests: Total failures: 1073
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3350363
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 166 edm output root files, 49 DQM output files
TriggerResults: no differences found

1) Merge files src/alpaka/classes_serial{.h,_def.xml} into src/classes{.h,_def.xml} and update BuildFile.xml 2) Split class definitions into two files, undoing commit 42fd4a7. This is possible due to PR cms-sw/cmsdist/pull/8705. Switching to this version required a small change to the Python configuration. These two changes were suggested for PR cms-sw#42807.

…package 1) Allocate buffers used for PFRecHitProducerKernel per-event. This allows making the producer global. 2) Change ::Grids synchronisation to ::Blocks synchronisation for GPU backend. This removes the need to distinguish between CPU and GPU backends in the kernel code. 3) Simplify code; add comments; more meaningful names; adapt CMSSW code style These two changes were suggested for PR cms-sw#42807.

mandrenguyen · 2023-10-26T08:13:28Z

+1

cmsbuild · 2023-10-26T08:13:52Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

antoniovilela · 2023-10-27T13:11:30Z

+1

cmsbuild added this to the CMSSW_13_3_X milestone Sep 18, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending new-package-pending code-checks-pending labels Sep 18, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 18, 2023

fllor added a commit to fllor/cms-bot that referenced this pull request Sep 18, 2023

Add RecoParticleFlow/PFRecHitProducer package

6edc81e

Required for CMSSW PR cms-sw/cmssw#42807

fllor mentioned this pull request Sep 18, 2023

Add RecoParticleFlow/PFRecHitProducer package cms-sw/cms-bot#2070

Merged

cmsbuild added the heterogeneous-pending label Sep 18, 2023

makortel mentioned this pull request Sep 18, 2023

Interplay of LCG_DICT_HEADER and LCG_DICT_XML with ALPAKA_BACKENDS #40846

Closed

makortel reviewed Sep 18, 2023

View reviewed changes

cmsbuild added pf and removed new-package-pending labels Sep 27, 2023

cmsbuild added tests-started heterogeneous-approved and removed tests-pending heterogeneous-pending labels Oct 25, 2023

cmsbuild added tests-approved and removed tests-started labels Oct 25, 2023

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Oct 26, 2023

jsamudio added a commit to jsamudio/cmssw that referenced this pull request Oct 26, 2023

Add changes from cms-sw#42807

15de704

jsamudio mentioned this pull request Oct 27, 2023

Add Alpaka Implementation of PFClusterProducer #43130

Merged

cmsbuild added orp-approved and removed orp-pending labels Oct 27, 2023

cmsbuild merged commit 7012a34 into cms-sw:master Oct 27, 2023

cmsbuild mentioned this pull request Oct 27, 2023

[LLVM] Update to version 17.0.3 cms-sw/cmsdist#8791

Merged

swagata87 mentioned this pull request Oct 30, 2023

Adapt PF code to read Hcal thresholds from GT #43025

Merged

malbouis mentioned this pull request Nov 17, 2023

Overview of implementations to consume HCalPFCuts from DB #43312

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Alpaka implementation of PFRecHitProducer #42807

Add Alpaka implementation of PFRecHitProducer #42807

fllor commented Sep 18, 2023 •

edited

Loading

cmsbuild commented Sep 18, 2023

cmsbuild commented Sep 18, 2023

fllor commented Sep 18, 2023

makortel commented Sep 18, 2023

cmsbuild commented Sep 18, 2023

hatakeyamak commented Sep 18, 2023

makortel Sep 18, 2023

swagata87 commented Sep 19, 2023

swagata87 commented Sep 27, 2023

hatakeyamak commented Oct 16, 2023 •

edited

Loading

hatakeyamak commented Oct 16, 2023

fwyzard commented Oct 17, 2023

fwyzard commented Oct 17, 2023 •

edited

Loading

cmsbuild commented Oct 25, 2023

fllor commented Oct 25, 2023

fwyzard commented Oct 25, 2023

fwyzard commented Oct 25, 2023

hatakeyamak commented Oct 25, 2023

fwyzard commented Oct 25, 2023

cmsbuild commented Oct 25, 2023

mandrenguyen commented Oct 26, 2023

cmsbuild commented Oct 26, 2023

antoniovilela commented Oct 27, 2023

Add Alpaka implementation of PFRecHitProducer #42807

Add Alpaka implementation of PFRecHitProducer #42807

Conversation

fllor commented Sep 18, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Sep 18, 2023

cmsbuild commented Sep 18, 2023

fllor commented Sep 18, 2023

makortel commented Sep 18, 2023

cmsbuild commented Sep 18, 2023

hatakeyamak commented Sep 18, 2023

makortel Sep 18, 2023

Choose a reason for hiding this comment

swagata87 commented Sep 19, 2023

swagata87 commented Sep 27, 2023

hatakeyamak commented Oct 16, 2023 • edited Loading

hatakeyamak commented Oct 16, 2023

fwyzard commented Oct 17, 2023

fwyzard commented Oct 17, 2023 • edited Loading

cmsbuild commented Oct 25, 2023

fllor commented Oct 25, 2023

fwyzard commented Oct 25, 2023

fwyzard commented Oct 25, 2023

hatakeyamak commented Oct 25, 2023

fwyzard commented Oct 25, 2023

cmsbuild commented Oct 25, 2023

Comparison Summary

mandrenguyen commented Oct 26, 2023

cmsbuild commented Oct 26, 2023

antoniovilela commented Oct 27, 2023

fllor commented Sep 18, 2023 •

edited

Loading

hatakeyamak commented Oct 16, 2023 •

edited

Loading

fwyzard commented Oct 17, 2023 •

edited

Loading