[12_4_X] Add GEM alignment in all Run3 GTs #38693

francescobrivio · 2022-07-11T21:42:08Z

PR description:

(Partial) Backport of #38692
In this PR we add:

the new GEM Alignment and AlignmentErrors tags to all Run3 GTs (CMSTalk post)
~~- the new BeamSpot for Run 3 collisions at 13.6 TeV in all Run3 MC GTs~~
~~- Compatible with the VtxSmearing parameters Realistic25ns13p6TeVEarly2022Collision introduced in Add VtxSmearing scenario for Run 3 early data at 13.6 TeV #38616~~
~~- The new reco BeamSpot was derived from the RelVal produced in JIRA-PDMVRELVALS-156~~
new DroboxMetaData tag for the Express and Prompt frozen GTs and in the data and data_relval GTs (CMSTalk post)

See master PR for details on the tags and the GT diffs.

EDIT
After the disussion in #38760 (comment) it was concluded that the new reco BeamSpot tags should not be propagated to the Run 3 realistic MC GTs in CMSSW_12_4_X (while they are included in the master branch).
So in aa754c5 I reverted the BeamSpot tags, here are the new GT diffs:
MC Run3 realistic https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_v8/124X_mcRun3_2022_realistic_v10
MC Run3 cosmics realistic https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022cosmics_realistic_deco_v9/124X_mcRun3_2022cosmics_realistic_deco_v11
MC Run3 realistic HI https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_HI_v8/124X_mcRun3_2022_realistic_HI_v10
MC Run3 2023 realistic https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2023_realistic_v9/124X_mcRun3_2023_realistic_v11
MC Run3 2024 realistic https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2024_realistic_v9/124X_mcRun3_2024_realistic_v11

PR validation:

Tested with
runTheMatrix.py -l 139.001,138.4,138.5,12034.0,11634.0,7.23,159.0,12434.0,12834.0 --ibeos -j 16

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #38692

francescobrivio · 2022-07-11T21:42:24Z

backport of #38692

cmsbuild · 2022-07-11T21:42:32Z

A new Pull Request was created by @francescobrivio for CMSSW_12_4_X.

It involves the following packages:

Configuration/AlCa (alca)

@malbouis, @yuanchao, @cmsbuild, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Add GEM alignment in all Run3 GTs and update BeamSpot in Run3 MC GTs #38692

francescobrivio · 2022-07-11T21:56:43Z

test parameters:

pull_request = [12_4_X] Run3 GEMGeometryBuilder Run3 modifier and fix DQM GEM chambers #38694
workflows = 12034.0,11634.0,7.23,159.0,12434.0,12834.0

cmsbuild · 2022-07-12T13:03:10Z

Pull request #38693 was updated. @malbouis, @yuanchao, @cmsbuild, @francescobrivio, @ChrisMisan, @tvami can you please check and sign again.

francescobrivio · 2022-07-12T13:27:54Z

@cmsbuild please test

tvami · 2022-07-12T15:10:24Z

type gem

cmsbuild · 2022-07-12T19:12:14Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba7c10/26167/summary.html
COMMIT: 18c24ed
CMSSW: CMSSW_12_4_X_2022-07-12-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38693/26167/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-mutracking_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-pixel_dqm_sourceclient had ERRORS

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 24570 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 4146357
DQMHistoTests: Total failures: 67138
DQMHistoTests: Total nulls: 7
DQMHistoTests: Total successes: 4079190
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 2.254 KiB( 53 files compared)
DQMHistoSizes: changed ( 11834.0 ): 2.250 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 227 log files, 45 edm output root files, 54 DQM output files
TriggerResults: found differences in 8 / 53 workflows

cmsbuild · 2022-07-12T21:34:35Z

Pull request #38693 was updated. @malbouis, @yuanchao, @cmsbuild, @francescobrivio, @ChrisMisan, @tvami can you please check and sign again.

francescobrivio · 2022-07-12T21:41:36Z

test parameters:

pull_request = [12_4_X] Run3 GEMGeometryBuilder Run3 modifier and fix DQM GEM chambers #38694,[12_4_X] Update GT for DQM clients unitTests and input file used for unitTest #38713
workflows = 12034.0,11634.0,7.23,159.0,12434.0,12834.0

francescobrivio · 2022-07-12T21:43:58Z

@cmsbuild please test

cmsbuild · 2022-07-15T18:49:50Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba7c10/26264/summary.html
COMMIT: ff54bde
CMSSW: CMSSW_12_4_X_2022-07-15-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38693/26264/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 24565 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 4146357
DQMHistoTests: Total failures: 67124
DQMHistoTests: Total nulls: 7
DQMHistoTests: Total successes: 4079204
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 2.254 KiB( 53 files compared)
DQMHistoSizes: changed ( 11834.0 ): 2.250 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 227 log files, 45 edm output root files, 54 DQM output files
TriggerResults: found differences in 8 / 53 workflows

cmsbuild · 2022-07-19T12:57:17Z

Pull request #38693 was updated. @malbouis, @yuanchao, @cmsbuild, @francescobrivio, @ChrisMisan, @tvami can you please check and sign again.

francescobrivio · 2022-07-19T12:59:20Z

hold

* the BS update is incompatible with the default vtx smearing in the upgrade wfs . Either update the PR or leave it out.

Following the discussion in #38760 (comment) I reverted the reco BeamSpot tags in the Run 3 MC realistic GTs in aa754c5.
I will update the PR description accordingly.

francescobrivio · 2022-07-19T13:07:30Z

@cmsbuild please test

mmusich · 2022-07-19T13:33:46Z

unhold

based on [12_4_X] Add GEM alignment in all Run3 GTs #38693 (comment)

cmsbuild · 2022-07-19T17:55:59Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ba7c10/26326/summary.html
COMMIT: aa754c5
CMSSW: CMSSW_12_4_X_2022-07-19-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38693/26326/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ba7c10/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 54
DQMHistoTests: Total histograms compared: 4146456
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 4146420
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 53 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 227 log files, 45 edm output root files, 54 DQM output files
TriggerResults: no differences found

tvami · 2022-07-20T09:01:04Z

+alca

PR according to the description (well its latest version, no beamspot changes after all)
diffs in Jenkins are known and not related

cmsbuild · 2022-07-20T09:01:22Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-07-20T17:39:29Z

@francescobrivio is it correct that the "design" tags are the same as in #38692? I imagine that that the vtxsmearing was not modified for those "design" scenarios, but let me double check...

francescobrivio · 2022-07-20T17:53:47Z

@francescobrivio is it correct that the "design" tags are the same as in #38692? I imagine that that the vtxsmearing was not modified for those "design" scenarios, but let me double check...

@perrotta correct! The "ideal" GTs use a different BS tag!

perrotta · 2022-07-20T18:00:51Z

+1

Ad GEM alignment in all Run3 GTs and update BeamSpot in Run3 MC GTs

443d702

cmsbuild added this to the CMSSW_12_4_X milestone Jul 11, 2022

cmsbuild added alca-pending backport orp-pending pending-signatures tests-pending labels Jul 11, 2022

francescobrivio mentioned this pull request Jul 11, 2022

[12_4_X] Run3 GEMGeometryBuilder Run3 modifier and fix DQM GEM chambers #38694

Merged

cmsbuild added the requires-external label Jul 11, 2022

update snapshot of Run3 data GTs

18c24ed

cmsbuild added tests-started and removed tests-pending labels Jul 12, 2022

cmsbuild added the gem label Jul 12, 2022

cmsbuild added tests-rejected and removed tests-started labels Jul 12, 2022

update run3_data GTs with correct GEMRECO_Geometry tag

ff54bde

cmsbuild added tests-pending and removed tests-rejected labels Jul 12, 2022

francescobrivio mentioned this pull request Jul 12, 2022

[12_4_X] Update GT for DQM clients unitTests and input file used for unitTest #38713

Merged

cmsbuild removed the tests-pending label Jul 12, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 15, 2022

revert reco BeamSpot tags in Run 3 realistic MC GTs

aa754c5

cmsbuild added tests-pending and removed tests-approved labels Jul 19, 2022

francescobrivio changed the title ~~[12_4_X] Add GEM alignment in all Run3 GTs and update BeamSpot in Run3 MC GTs~~ [12_4_X] Add GEM alignment in all Run3 GTs Jul 19, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 19, 2022

cmsbuild removed the hold label Jul 19, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 19, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jul 20, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 20, 2022

cmsbuild merged commit 5a556c5 into cms-sw:CMSSW_12_4_X Jul 20, 2022

francescobrivio mentioned this pull request Jul 22, 2022

Add GEM alignment in all Run3 GTs #38832

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[12_4_X] Add GEM alignment in all Run3 GTs #38693

[12_4_X] Add GEM alignment in all Run3 GTs #38693

francescobrivio commented Jul 11, 2022 •

edited

Loading

francescobrivio commented Jul 11, 2022

cmsbuild commented Jul 11, 2022 •

edited

Loading

francescobrivio commented Jul 11, 2022

cmsbuild commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

tvami commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

cmsbuild commented Jul 15, 2022

cmsbuild commented Jul 19, 2022

francescobrivio commented Jul 19, 2022 •

edited

Loading

francescobrivio commented Jul 19, 2022

mmusich commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

tvami commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 20, 2022

francescobrivio commented Jul 20, 2022

perrotta commented Jul 20, 2022

[12_4_X] Add GEM alignment in all Run3 GTs #38693

[12_4_X] Add GEM alignment in all Run3 GTs #38693

Conversation

francescobrivio commented Jul 11, 2022 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

francescobrivio commented Jul 11, 2022

cmsbuild commented Jul 11, 2022 • edited Loading

francescobrivio commented Jul 11, 2022

cmsbuild commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

tvami commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

Unit Tests

Comparison Summary

cmsbuild commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

francescobrivio commented Jul 12, 2022

cmsbuild commented Jul 15, 2022

Comparison Summary

cmsbuild commented Jul 19, 2022

francescobrivio commented Jul 19, 2022 • edited Loading

francescobrivio commented Jul 19, 2022

mmusich commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

Comparison Summary

tvami commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 20, 2022

francescobrivio commented Jul 20, 2022

perrotta commented Jul 20, 2022

francescobrivio commented Jul 11, 2022 •

edited

Loading

cmsbuild commented Jul 11, 2022 •

edited

Loading

francescobrivio commented Jul 19, 2022 •

edited

Loading