Customize concurrency for ESSource DataProxies

[Dr15Jones]

PR description:

Refactored ESSource specific DataProxies to allow customizing how concurrency should be handled. Provided implementations to support either concurrent or non-concurrent running of DataProxies for the same ESSource.

This was ability will be used by PoolDBESSource.

PR validation:

Code compiles and all framework unit tests (including a new one) all succeed.

closes #36487

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36539/27449

• This PR adds an extra 44KB to repository

[cmsbuild]

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

• FWCore/Framework (core)
• FWCore/Integration (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[Dr15Jones]

please test

[cmsbuild]

-1

Failed Tests: Build ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e2acfe/21363/summary.html
COMMIT: d6e96d7
CMSSW: CMSSW_12_3_X_2021-12-17-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36539/21363/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @hftsoi Calo-L1 unpacker modified to read 5BX payload format #36259

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e2acfe/21363/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e2acfe/21363/git-merge-result

Build

I found compilation error when building:

/cvmfs/cms-ib.cern.ch/nweek-02711/slc7_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/include/c++/10.3.0/bits/shared_ptr.h:408:59:   required from 'std::shared_ptr<_Tp>::shared_ptr(std::_Sp_alloc_shared_tag<_Tp>, _Args&& ...) [with _Alloc = std::allocator > >; _Args = {std::shared_ptr >&, edm::SerialTaskQueue*&, std::mutex*&}; _Tp = DataProxy >]'
/cvmfs/cms-ib.cern.ch/nweek-02711/slc7_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/include/c++/10.3.0/bits/shared_ptr.h:859:14:   required from 'std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = DataProxy >; _Alloc = std::allocator > >; _Args = {std::shared_ptr >&, edm::SerialTaskQueue*&, std::mutex*&}]'
/cvmfs/cms-ib.cern.ch/nweek-02711/slc7_amd64_gcc10/external/gcc/10.3.0-84898dea653199466402e67d73657f10/include/c++/10.3.0/bits/shared_ptr.h:875:39:   required from 'std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = DataProxy >; _Args = {std::shared_ptr >&, edm::SerialTaskQueue*&, std::mutex*&}]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/CondCore/ESSources/interface/DataProxy.h:152:55:   required from 'void DataProxyWrapper::lateInit(cond::persistency::Session&, const string&, const boost::posix_time::ptime&, const string&, const string&, edm::SerialTaskQueue*, std::mutex*) [with RecordT = CSCPedestalsRcd; DataT = CSCPedestals; Initializer = cond::DefaultInitializer; std::string = std::__cxx11::basic_string]'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/CondCore/CSCPlugins/src/plugin.cc:83:1:   required from here
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/FWCore/Framework/interface/ESSourceDataProxyTemplate.h:39:47: error: type 'edm::eventsetup::ESSourceDataProxyBase' is not a direct base of 'edm::eventsetup::ESSourceDataProxyTemplate'
   39 |         : ESSourceDataProxyBase(iQueue, iMutex) {}
      |                                               ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/FWCore/Framework/interface/ESSourceDataProxyTemplate.h:39:47: error: no matching function for call to 'edm::eventsetup::ESSourceDataProxyNonConcurrentBase::ESSourceDataProxyNonConcurrentBase()'
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/FWCore/Framework/interface/ESSourceDataProxyTemplate.h:30,
                 from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_12_3_X_2021-12-17-1100/src/CondCore/ESSources/interface/DataProxy.h:11,


Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36539/27455

• This PR adds an extra 44KB to repository

[cmsbuild]

Pull request #36539 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

[Dr15Jones]

please test

[cmsbuild]

Pull request #36539 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36539/27498

• This PR adds an extra 20KB to repository

[cmsbuild]

Pull request #36539 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @santocch, @makortel can you please check and sign again.

[Dr15Jones]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e2acfe/21411/summary.html
COMMIT: 01aa32e
CMSSW: CMSSW_12_3_X_2021-12-20-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36539/21411/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 43
• DQMHistoTests: Total histograms compared: 3461688
• DQMHistoTests: Total failures: 11
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3461655
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 42 files compared)
• Checked 181 log files, 42 edm output root files, 43 DQM output files
• TriggerResults: no differences found

[makortel]

+1

[perrotta]

+1

[perrotta]

merge

[santocch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.