CUDA, ROCm and Alpaka-related updates [13.0.x]

[fwyzard]

PR description:

CUDA, ROCm and Alpaka-related updates:

• add PortableTestObjects data formats for the ROCm Alpaka backend
• add support for AMD GPUs via the ProcessAcceleratorROCm
• update the description of the "backend" parameter
• introduce different exit status codes for cudaIsEnabled and rocmIsEnabled
• add abstract interfaces for the CUDA and ROCm services
• do not load the CUDA or ROCm services if the backends are not available

PR validation:

All unit tests should pass.

Backport status

(Partial) backport of #40832 to CMSSW 13.0.x for data taking.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40725/34123

• This PR adds an extra 20KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlpakaProducer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285

[cmsbuild]

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

• DataFormats/PortableTestObjects (heterogeneous)
• HeterogeneousCore/AlpakaCore (heterogeneous)
• HeterogeneousCore/AlpakaTest (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@makortel, @missirol, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from CUDA, ROCm and Alpaka-related updates #40832

[fwyzard]

@ericcano @makortel it looks like ROCm does not have an exact equivalent of CUDA's __trap(); the closest are abort() and __builtin_trap() (the former calls the latter), which make the host program abort:

===== Test "testRocmSoALayoutAndView_t" ====
Pass: expected range-check exception (operator[]) successfully caught on the host.
Pass: expected range-check exception (view-level index access) successfully caught on the host.
:0:rocdevice.cpp            :2673: 2440641366144 us: 3392928: [tid:0x7fd788faa700] Device::callbackQueue aborting with error : HSA_STATUS_ERROR_EXCEPTION: An HSAIL operation resulted in a hardware exception. code: 0x1016
/bin/sh: line 1: 3392928 Aborted                 (core dumped) sh -c 'testRocmSoALayoutAndView_t '

---> test testRocmSoALayoutAndView_t had ERRORS
TestTime:1
^^^^ End Test testRocmSoALayoutAndView_t ^^^^

This introduces a different behaviour between the CPU, CUDA, and ROCm backends for the range-check accessors: the CPU and CUDA bakends raise an execption, while ROCm causes the program to abort.

What do you think ?

[fwyzard]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40725/34128

• This PR adds an extra 32KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlpakaProducer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/test/writer.py modified in PR(s): Add support to multi layout portable collections (any number) #40285

[cmsbuild]

Pull request #40725 was updated. @perrotta, @rappoccio, @makortel, @fwyzard, @fabiocos, @davidlange6 can you please check and sign again.

[fwyzard]

please test

[fwyzard]

please test for el8_ppc64le_gcc11

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40725/34129

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File HeterogeneousCore/AlpakaTest/plugins/alpaka/TestAlpakaProducer.cc modified in PR(s): Add support to multi layout portable collections (any number) #40285
  • File HeterogeneousCore/AlpakaTest/test/writer.py modified in PR(s): Add support to multi layout portable collections (any number) #40285

[fwyzard]

thanks @mandrenguyen - could you sign also #40832, which is the 13.1.x equivalent of this PR ?

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-69a589/30950/summary.html
COMMIT: 5ebe8d8
CMSSW: CMSSW_13_0_X_2023-02-28-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40725/30950/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 11 lines from the logs
• Reco comparison results: 9 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3557190
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3557162
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[AdrianoDee]

+upgrade
(upgrade-wise only changes to few headers)

[perrotta]

@fwyzard while this backport touches 72 files, the PR just merged in master touched 81 of them.
To help us digging through several hundred changed lines of code, could you please summarize, possibly also in the PR description, the differences between this PR and #40832?

[fwyzard]

Of course.
The 13.1.x version includes all the changes in the 13.0.x one (they are based on the same commit, 5ebe8d8).

The 13.1.x version has an additional set of changes to address files that were introduced only in master for PhysicsTools/TensorFlow:

• adc5096
• 1c30d1c

Those were not backported, because the corresponding files are not in 13.0.x .

[fwyzard]

I've added an extra item only to the description of #40832:

• update the PhysicsTools/TensorFlow tests accordingly

[perrotta]

Thank you @fwyzard

[rappoccio]

@makortel there are a few comments that are still unresolved, but everything is signed here. Are we good to merge?

[makortel]

I'm good

[rappoccio]

+1

• Will include in 13.0

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will be automatically merged.