Hcal Phase I #2265

ianna · 2014-02-02T16:36:44Z

Merging all branches
Fix from Sunanda for Phase I

…y are taken from Hcal numbering now.

cmsbuild · 2014-02-02T16:37:50Z

A new Pull Request was created by @ianna for CMSSW_6_2_X_SLHC.

Hcal Phase I

It involves the following packages:

CalibCalorimetry/CaloMiscalibTools
CalibCalorimetry/HcalAlgos
CalibCalorimetry/HcalPlugins
CalibCalorimetry/HcalStandardModules
CalibCalorimetry/HcalTPGEventSetup
CalibFormats/HcalObjects
Calibration/HcalCalibAlgos
Calibration/IsolatedParticles
CaloOnlineTools/HcalOnlineDb
CondFormats/DataRecord
CondTools/Geometry
CondTools/Hcal
Configuration/Geometry
DQM/HcalMonitorClient
DQM/HcalMonitorTasks
FastSimulation/CaloRecHitsProducer
FastSimulation/L1CaloTriggerProducer
Geometry/CMSCommonData
Geometry/CaloEventSetup
Geometry/CaloTopology
Geometry/HcalCommonData
Geometry/HcalEventSetup
Geometry/HcalTowerAlgo
Geometry/Records
L1TriggerConfig/L1ScalesProducers
RecoEgamma/EgammaIsolationAlgos
RecoJets/JetProducers
RecoLocalCalo/CaloRecCandCreator
RecoLocalCalo/CaloTowersCreator
RecoLocalCalo/HcalRecProducers
RecoParticleFlow/PFClusterProducer
SimCalorimetry/HcalSimAlgos
SimCalorimetry/HcalSimProducers
SimCalorimetry/HcalTrigPrimProducers

@diguida, @deguio, @lveldere, @ianna, @anton-a, @thspeer, @rcastello, @civanch, @ojeda, @cmsbuild, @ktf, @Dr15Jones, @mdhildreth, @rovere, @giamman, @slava77, @ggovi, @Degano, @mulhearn, @apfeiffer1, @nclopezo, @danduggan can you please review it and eventually sign? Thanks.
@ghellwig, @yslai, @TaiSakuma this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@andersonjacob, @mark-grimes you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

ianna · 2014-02-02T16:42:46Z

+1

tested with

runTheMatrix.py --what upgrade -l 3300

all steps pass.

mark-grimes · 2014-02-02T18:19:26Z

Full test suite still running, but testing the GEMs with

runTheMatrix.py -w upgrade -l 3400 --command "--customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019WithGem”

fails when customising:

customising the process with cust_2019WithGem from SLHCUpgradeSimulations/Configuration/combinedCustoms
Traceback (most recent call last):
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/bin/slc5_amd64_gcc472/cmsDriver.py", line 43, in <module>
    run()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/bin/slc5_amd64_gcc472/cmsDriver.py", line 15, in run
    configBuilder.prepare()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/python/Configuration/Applications/ConfigBuilder.py", line 2082, in prepare
    self.pythonCfgCode += self.addCustomise()
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/python/Configuration/Applications/ConfigBuilder.py", line 795, in addCustomise
    self.process=getattr(package,fcn)(self.process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/python/SLHCUpgradeSimulations/Configuration/combinedCustoms.py", line 62, in cust_2019WithGem
    process=cust_2019(process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/python/SLHCUpgradeSimulations/Configuration/combinedCustoms.py", line 57, in cust_2019
    process=customise_HcalPhase1(process)
  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/python/SLHCUpgradeSimulations/Configuration/HCalCustoms.py", line 61, in customise_HcalPhase1
    process.es_hardcode.hcalTopologyConstants.mode=cms.string('HcalTopologyMode::SLHC')
AttributeError: 'ESSource' object has no attribute 'hcalTopologyConstants'

I'll report on the other tests when they're finished.

ianna · 2014-02-02T18:23:58Z

@mark-grimes - customisations are not needed for Hcal.

process.es_hardcode.hcalTopologyConstants

should be removed from where it is defned.

ianna · 2014-02-03T06:48:25Z

@mark-grimes - this should fix the 3400 workflow.

@salavat - could you, please, check customisations for Hcal? I think, I've removed everything unneeded.

cmsbuild · 2014-02-03T06:52:51Z

Pull request #2265 was updated. @diguida, @deguio, @lveldere, @ianna, @anton-a, @thspeer, @rcastello, @civanch, @ojeda, @cmsbuild, @ktf, @Dr15Jones, @mdhildreth, @rovere, @giamman, @slava77, @ggovi, @Degano, @mulhearn, @apfeiffer1, @nclopezo, @danduggan can you please check and sign again.

cmsbuild · 2014-02-03T07:07:44Z

Pull request #2265 was updated. @diguida, @deguio, @lveldere, @ianna, @anton-a, @thspeer, @rcastello, @civanch, @ojeda, @cmsbuild, @ktf, @Dr15Jones, @mdhildreth, @rovere, @giamman, @slava77, @ggovi, @Degano, @mulhearn, @apfeiffer1, @nclopezo, @danduggan can you please check and sign again.

ianna · 2014-02-03T07:08:18Z

+1

lveldere · 2014-02-03T08:06:31Z

+1

mark-grimes · 2014-02-03T10:44:07Z

This fixes the configuration problem (3400, 4400, 4500, 4502 and 15001 also failed before this fix). Now 3400 and 15001 fail in harvesting with:

----- Begin Fatal Exception 03-Feb-2014 11:02:47 CET-----------------------
An exception of category 'Conditions mismatch' occurred while
   [0] Processing run: 1
   [1] Running path 'dqmHarvesting'
   [2] Calling beginRun for module HcalMonitorClient/'hcalClient'
Exception Message:
Requested conditions of type HcalPedestals for cell (0x42006101) (HB 2,1,1) got conditions for cell (0x4200608d) (HB 1,13,1)
----- End Fatal Exception -------------------------------------------------

Note that there are several of these messages in the log:

%MSG-e HCAL:  HcalMonitorClient:hcalClient@beginRun 03-Feb-2014 11:06:07 CET  Run: 1
Inconsistent dense packing between current topology (10) and calibration object (0)

3400 when running with combinedCustoms.cust_2019WithGem crashes in harvesting with the following stack trace:

%MSG-e FatalSystemSignal:  HcalDigisClient:hcaldigisClient@endRun  03-Feb-2014 11:03:34 CET End Run: 1
A fatal system signal has occurred: segmentation violation

<trimmed>

#5  0x00002b8b696e3a74 in MonitorElement::Fill(double, double) () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libDQMServicesCore.so
#6  0x00002b8b79128cdc in HcalDigisClient::HcalDigisEndjob(std::vector<MonitorElement*, std::allocator<MonitorElement*> > const&, std::string) () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginValidationHcalDigis.so
#7  0x00002b8b79129878 in HcalDigisClient::runClient() () from /tmp/grimes/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/pluginValidationHcalDigis.so
#8  0x00002b8b5a8e9330 in edm::EDAnalyzer::doEndRun(edm::RunPrincipal const&, edm::EventSetup const&, edm::CurrentProcessingContext const*) () from /afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc5_amd64_gcc472/cms/cmssw/CMSSW_6_2_X_SLHC_2014-02-02-1400/lib/slc5_amd64_gcc472/libFWCoreFramework.so

cmsbuild · 2014-02-03T11:43:15Z

Pull request #2265 was updated. @diguida, @deguio, @lveldere, @ianna, @anton-a, @thspeer, @rcastello, @civanch, @ojeda, @cmsbuild, @ktf, @Dr15Jones, @mdhildreth, @rovere, @giamman, @slava77, @ggovi, @Degano, @mulhearn, @apfeiffer1, @nclopezo, @danduggan can you please check and sign again.

ianna · 2014-02-03T11:47:46Z

@mark-grimes - strange, the following command runs fine for me:

runTheMatrix.py --what upgrade -l 3400 --command "--customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019WithGem"

ianna · 2014-02-03T14:05:04Z

+1

mark-grimes · 2014-02-03T14:50:38Z

Tried again with the updated pull request. The GEM test (as in @ianna's comment above) now runs fine but still get the same errors in harvesting for tests 3400 and 15001.

ianna · 2014-02-03T14:52:45Z

@mark-grimes - is it step4 your talking about?

mark-grimes · 2014-02-03T14:55:58Z

@ianna Yes, step 4 for both 3400 and 15001.

Tests 3300, 4100, 4400, 40001, 50002, 60002, 60001, 4502, 4500, 5001, and 50001 run fine all the way through.

ianna · 2014-02-03T16:47:09Z

@mark-grimes - I guess, the errors come when you do not have additional (correct?) customize option. Harvesting fails due to collection name mismatch which is defined in customization fragments for all other steps. Customization of step4 does not include "--customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019" as the other steps do, there is only tracker customization.

diff 3400_FourMuPt1_200+FourMuPt1_200_UPG2019+DIGIUP19+RECOUP19+HARVESTUP19/step4_HARVESTING.py ../relval/3400_FourMuPt1_200+FourMuPt1_200_UPG2019+DIGIUP19+RECOUP19+HARVESTUP19/.
5c5
< # with command line options: step4 --conditions auto:upgrade2019 -s HARVESTING:validationHarvesting+dqmHarvesting --customise SLHCUpgradeSimulations/Configuration/phase1TkCustoms.customise --geometry Extended2019 --mc --magField 38T_PostLS1 -n 100 --filein file:step3_inDQM.root --fileout file:step4.root
---
> # with command line options: step4 --conditions auto:upgrade2019 -s HARVESTING:validationHarvesting+dqmHarvesting --customise SLHCUpgradeSimulations/Configuration/phase1TkCustoms.customise --geometry Extended2019 --mc --magField 38T_PostLS1 --customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019WithGem -n 100 --filein file:step3_inDQM.root --fileout file:step4.root
70,71c70,71
< # Automatic addition of the customisation function from SLHCUpgradeSimulations.Configuration.phase1TkCustoms
< from SLHCUpgradeSimulations.Configuration.phase1TkCustoms import customise 
---
> # Automatic addition of the customisation function from SLHCUpgradeSimulations.Configuration.combinedCustoms
> from SLHCUpgradeSimulations.Configuration.combinedCustoms import cust_2019WithGem 
73,74c73,74
< #call to customisation function customise imported from SLHCUpgradeSimulations.Configuration.phase1TkCustoms
< process = customise(process)
---
> #call to customisation function cust_2019WithGem imported from SLHCUpgradeSimulations.Configuration.combinedCustoms
> process = cust_2019WithGem(process)

ianna · 2014-02-03T16:49:45Z

@mark-grimes - correction - hcal conditions mismatch

cmsbuild · 2014-02-03T20:59:18Z

-1
I ran the usual tests and I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2265/16/summary.html

ianna · 2014-02-04T07:56:40Z

@cmsbuild - the test which fails is not applicable to SLHC. It test LHC startup scenario:
https://github.com/cms-sw/cmssw/blob/CMSSW_6_2_X_SLHC/PhysicsTools/PatAlgos/python/patTemplate_cfg.py

mark-grimes · 2014-02-04T10:36:52Z

merge

@ianna yes, harvesting only has the tracking customisations. Tried step4 of 3400 with the full customisation and it ran through.

Hcal Phase I

davidlange6 · 2014-02-04T10:55:28Z

did we then update 3400 to have the full customization in step4? (this can be generally fixed I guess - indeed was a problem before)

On Feb 4, 2014, at 11:36 AM, Mark Grimes notifications@github.com
wrote:

merge

@ianna yes, harvesting only has the tracking customisations. Tried step4 of 3400 with the full customisation and it ran through.

—
Reply to this email directly or view it on GitHub.

ianna · 2014-02-04T11:02:48Z

@davidlange6 - or we see if Hcal relabeling settings can be moved out. I'll check it when the new IB is ready.

Sunanda and others added 21 commits January 24, 2014 17:05

Relabelling with new Rec numbering Record

dd97dc8

Topology and geometry built from RecNumbering Record

e43100b

Delete an obsolete file

c3c9ec9

Update the geometry cff files to initiate a producer

b8d9fec

Update the geometry cff files to initiate a producer

7a9fa31

Update the geometry cff files to initiate a producer

dfeeabc

Merged Phase2HE_9 from repository bsunanda

e2e6426

Merged Phase2HE_10 from repository bsunanda

055219f

Merged Phase2-HE12 from repository bsunanda

e9c161a

Remove verbosity

f53fc28

Merged Phase2HE_11 from repository bsunanda

e068622

Further changes to use new HcalTopology

9bc9716

Merged Phase2HE_14 from repository bsunanda

b438331

Update HcalTopology record which now depends on HcalRecNumbering.

f1b0363

Update HcalTopology record which now depends on HcalRecNumbering.

157964a

Remove Hcal Topology constants from configuration file fragments. The…

f6f0285

…y are taken from Hcal numbering now.

Replace IdealGeometryRecord with HcalRecNumberingRecord.

8762ac4

Add dependent numbering record to HcalTopology configuration.

db91d14

Correcting record dependency.

ec27110

Merged hcal-topology-and-numbering-records from repository ianna

ef4687f

Fixes for Hcal PhaseI - all steps pass.

cd396e0

ianna mentioned this pull request Feb 2, 2014

Hcal Topology and Numbering Records #2220

Closed

ianna added 2 commits February 3, 2014 07:44

Remove unneeded customizations for Hcal geometry.

dd643cd

Hcal topology is a rec numbering record now.

6ce3834

Add back digi relabelling.

097a60c

The relabelling constants come from a record.

594b472

cmsbuild added a commit that referenced this pull request Feb 4, 2014

Merge pull request #2265 from ianna/hcal-phaseI

bd7c0ca

Hcal Phase I

cmsbuild merged commit bd7c0ca into cms-sw:CMSSW_6_2_X_SLHC Feb 4, 2014

ianna deleted the hcal-phaseI branch October 26, 2015 16:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hcal Phase I #2265

Hcal Phase I #2265

ianna commented Feb 2, 2014

cmsbuild commented Feb 2, 2014

ianna commented Feb 2, 2014

mark-grimes commented Feb 2, 2014

ianna commented Feb 2, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

lveldere commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

ianna commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

ianna commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

ianna commented Feb 3, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

ianna commented Feb 4, 2014

Hcal Phase I #2265

Hcal Phase I #2265

Conversation

ianna commented Feb 2, 2014

cmsbuild commented Feb 2, 2014

ianna commented Feb 2, 2014

mark-grimes commented Feb 2, 2014

ianna commented Feb 2, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

lveldere commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 3, 2014

ianna commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

ianna commented Feb 3, 2014

mark-grimes commented Feb 3, 2014

ianna commented Feb 3, 2014

ianna commented Feb 3, 2014

cmsbuild commented Feb 3, 2014

ianna commented Feb 4, 2014

mark-grimes commented Feb 4, 2014

davidlange6 commented Feb 4, 2014

ianna commented Feb 4, 2014