Skip to content
This repository has been archived by the owner on Jan 31, 2022. It is now read-only.

Bug Report: gemSCurveAnaToolkit.py generates crash with ROOT v6.X or on cc7? #63

Open
1 of 2 tasks
bdorney opened this issue Jan 18, 2018 · 7 comments
Open
1 of 2 tasks

Bug Report: gemSCurveAnaToolkit.py generates crash with ROOT v6.X or on cc7? #63

bdorney opened this issue Jan 18, 2018 · 7 comments
Assignees
@jsturdy @bdorney @mexanick
Labels
Priority: Medium Status: Help Wanted Type: Bug

Comments

@bdorney
Copy link
Contributor

bdorney commented Jan 18, 2018 •
edited
Loading

Brief summary of issue

gemSCurveAnaToolkit.py generates a crash when running on cc7 with ROOT v6.X. Suspect it's a difference between ROOT 5.34 and 6.X, unclear why.

Types of issue

  • Bug report (report an issue with the code)
  • Feature request (request for change which adds functionality)

Expected Behavior

Current Behavior

On gem904daq03

Works

[dorney@gem904daq03]~/scratch0/CMS_GEM/CMS_GEM_DAQ/gem-plotting-tools% gemSCurveAnaToolkit.py -i listOfScanDates_Scurve.txt -c -s30 -v1 --anaType=scurveAna --drawLeg
TClass::TClass:0: RuntimeWarning: no dictionary for class TF1Parameters is available
TStreamerInfo::BuildOld:0: RuntimeWarning: Cannot convert TF1::fParErrors from type:vector<double> to type:Double_t*, skip element
TStreamerInfo::BuildOld:0: RuntimeWarning: Cannot convert TF1::fParMin from type:vector<double> to type:Double_t*, skip element
TStreamerInfo::BuildOld:0: RuntimeWarning: Cannot convert TF1::fParMax from type:vector<double> to type:Double_t*, skip element
TStreamerInfo::BuildOld:0: RuntimeWarning: Cannot convert TF1::fSave from type:vector<double> to type:Double_t*, skip element
TStreamerInfo::BuildOld:0: RuntimeWarning: Cannot convert TF1::fParams from type:TF1Parameters* to type:Double_t*, skip element
Info in <TCanvas::Print>: png file canv_SCurveFitOverlay_VFAT1_Chan30.png has been created
eog canv_SCurveFitOverlay_GE11-X-S-CERN-0003_2018.01.17.14.29_VFAT1_Chan30.png

Your plot is stored in a TFile, to open it execute:
root /afs/cern.ch/user/d/dorney/scratch0/CMS_GEM/CMS_GEM_DAQ/data/gemelog/gemSCurveAnaToolkit.root

ROOT is 5.34/36
python is Python 2.6.6

On gem904qc8daq

[dorney@gem904qc8daq]~/scratch0/CMS_GEM/CMS_GEM_DAQ/gem-plotting-tools% gemSCurveAnaToolkit.py -i listOfScanDates_Scurve.txt -c -s30 -v1 --anaType=scurveAna --drawLeg

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007fe88412bdbc in waitpid () from /lib64/libc.so.6
#1  0x00007fe8840aecc2 in do_system () from /lib64/libc.so.6
#2  0x00007fe87c3041a2 in TUnixSystem::StackTrace() () from /usr/lib64/root/libCore.so.6.10
#3  0x00007fe87c306b3c in TUnixSystem::DispatchSignals(ESignals) () from /usr/lib64/root/libCore.so.6.10
#4  <signal handler called>
#5  0x00007fe86f47289f in TF1::GetParameters() const () from /usr/lib64/root/libHist.so.6.10
#6  0x00007fe86f46904b in TF1::DoCreateHistogram(double, double, bool) () from /usr/lib64/root/libHist.so.6.10
#7  0x00007fe86f46889d in TF1::Paint(char const*) () from /usr/lib64/root/libHist.so.6.10
#8  0x00007fe86fbed5ac in TPad::Paint(char const*) () from /usr/lib64/root/libGpad.so.6.10.08
#9  0x00007fe86fbeb84f in TPad::Print(char const*, char const*) () from /usr/lib64/root/libGpad.so.6.10.08
#10 0x00007fe86fbec86f in TPad::SaveAs(char const*, char const*) const () from /usr/lib64/root/libGpad.so.6.10.08
#11 0x00007fe87ce79bd7 in FastCall(long, void*, void*, void*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#12 0x00007fe87ce7ea0f in PyROOT::TVoidExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#13 0x00007fe87cea239d in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#14 0x00007fe87cea0d29 in PyROOT::TMethodHolder::Execute(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#15 0x00007fe87ce9ff1e in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#16 0x00007fe87ce845ee in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#17 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#18 0x00007fe884e350f6 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#19 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#20 0x00007fe884e393fc in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#21 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#22 0x00007fe884e3c002 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
#23 0x00007fe884e5543f in run_mod () from /lib64/libpython2.7.so.1.0
#24 0x00007fe884e565fe in PyRun_FileExFlags () from /lib64/libpython2.7.so.1.0
#25 0x00007fe884e57889 in PyRun_SimpleFileExFlags () from /lib64/libpython2.7.so.1.0
#26 0x00007fe884e68a3f in Py_Main () from /lib64/libpython2.7.so.1.0
#27 0x00007fe88408ec05 in __libc_start_main () from /lib64/libc.so.6
#28 0x000000000040071e in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum.
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007fe86f47289f in TF1::GetParameters() const () from /usr/lib64/root/libHist.so.6.10
#6  0x00007fe86f46904b in TF1::DoCreateHistogram(double, double, bool) () from /usr/lib64/root/libHist.so.6.10
#7  0x00007fe86f46889d in TF1::Paint(char const*) () from /usr/lib64/root/libHist.so.6.10
#8  0x00007fe86fbed5ac in TPad::Paint(char const*) () from /usr/lib64/root/libGpad.so.6.10.08
#9  0x00007fe86fbeb84f in TPad::Print(char const*, char const*) () from /usr/lib64/root/libGpad.so.6.10.08
#10 0x00007fe86fbec86f in TPad::SaveAs(char const*, char const*) const () from /usr/lib64/root/libGpad.so.6.10.08
#11 0x00007fe87ce79bd7 in FastCall(long, void*, void*, void*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#12 0x00007fe87ce7ea0f in PyROOT::TVoidExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#13 0x00007fe87cea239d in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#14 0x00007fe87cea0d29 in PyROOT::TMethodHolder::Execute(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#15 0x00007fe87ce9ff1e in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#16 0x00007fe87ce845ee in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#17 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#18 0x00007fe884e350f6 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#19 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#20 0x00007fe884e393fc in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#21 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#22 0x00007fe884e3c002 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
#23 0x00007fe884e5543f in run_mod () from /lib64/libpython2.7.so.1.0
#24 0x00007fe884e565fe in PyRun_FileExFlags () from /lib64/libpython2.7.so.1.0
#25 0x00007fe884e57889 in PyRun_SimpleFileExFlags () from /lib64/libpython2.7.so.1.0
#26 0x00007fe884e68a3f in Py_Main () from /lib64/libpython2.7.so.1.0
#27 0x00007fe88408ec05 in __libc_start_main () from /lib64/libc.so.6
#28 0x000000000040071e in _start ()
===========================================================


Traceback (most recent call last):
  File "/afs/cern.ch/user/d/dorney/scratch0/CMS_GEM/CMS_GEM_DAQ/gem-plotting-tools/macros/gemSCurveAnaToolkit.py", line 90, in <module>
    vfatChNotROBstr=options.channels
  File "/afs/cern.ch/user/d/dorney/scratch0/CMS_GEM/CMS_GEM_DAQ/gem-plotting-tools/macros/scurvePlottingUtitilities.py", line 48, in overlay_scurve
    canvas.SaveAs("%s.png"%(canvas.GetName()))
SystemError: void TPad::SaveAs(const char* filename = "", const char* option = "") =>
    problem in C++; program state has been reset

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007fe88412bdbc in waitpid () from /lib64/libc.so.6
#1  0x00007fe8840aecc2 in do_system () from /lib64/libc.so.6
#2  0x00007fe87c3041a2 in TUnixSystem::StackTrace() () from /usr/lib64/root/libCore.so.6.10
#3  0x00007fe87c306b3c in TUnixSystem::DispatchSignals(ESignals) () from /usr/lib64/root/libCore.so.6.10
#4  <signal handler called>
#5  0x0000000000000000 in ?? ()
#6  0x00007fe87c288aba in TList::Delete(char const*) () from /usr/lib64/root/libCore.so.6.10
#7  0x00007fe87c17a1c8 in TROOT::EndOfProcessCleanups() () from /usr/lib64/root/libCore.so.6.10
#8  0x00007fe87ce79bd7 in FastCall(long, void*, void*, void*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#9  0x00007fe87ce7ea0f in PyROOT::TVoidExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#10 0x00007fe87cea239d in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#11 0x00007fe87cea0d29 in PyROOT::TMethodHolder::Execute(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#12 0x00007fe87ce9ff1e in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#13 0x00007fe87ce845ee in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#14 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#15 0x00007fe884e350f6 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#16 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#17 0x00007fe884dc594d in function_call () from /lib64/libpython2.7.so.1.0
#18 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#19 0x00007fe884e345bd in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#20 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#21 0x00007fe884dc5858 in function_call () from /lib64/libpython2.7.so.1.0
#22 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#23 0x00007fe884e327b7 in PyEval_CallObjectWithKeywords () from /lib64/libpython2.7.so.1.0
#24 0x00007fe884e57047 in Py_Finalize () from /lib64/libpython2.7.so.1.0
#25 0x00007fe884e68445 in Py_Main () from /lib64/libpython2.7.so.1.0
#26 0x00007fe88408ec05 in __libc_start_main () from /lib64/libc.so.6
#27 0x000000000040071e in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum.
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x0000000000000000 in ?? ()
#6  0x00007fe87c288aba in TList::Delete(char const*) () from /usr/lib64/root/libCore.so.6.10
#7  0x00007fe87c17a1c8 in TROOT::EndOfProcessCleanups() () from /usr/lib64/root/libCore.so.6.10
#8  0x00007fe87ce79bd7 in FastCall(long, void*, void*, void*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#9  0x00007fe87ce7ea0f in PyROOT::TVoidExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#10 0x00007fe87cea239d in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#11 0x00007fe87cea0d29 in PyROOT::TMethodHolder::Execute(void*, long, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#12 0x00007fe87ce9ff1e in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#13 0x00007fe87ce845ee in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /usr/lib64/python2.7/site-packages/libPyROOT.so
#14 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#15 0x00007fe884e350f6 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#16 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#17 0x00007fe884dc594d in function_call () from /lib64/libpython2.7.so.1.0
#18 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#19 0x00007fe884e345bd in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#20 0x00007fe884e3befd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#21 0x00007fe884dc5858 in function_call () from /lib64/libpython2.7.so.1.0
#22 0x00007fe884da09a3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#23 0x00007fe884e327b7 in PyEval_CallObjectWithKeywords () from /lib64/libpython2.7.so.1.0
#24 0x00007fe884e57047 in Py_Finalize () from /lib64/libpython2.7.so.1.0
#25 0x00007fe884e68445 in Py_Main () from /lib64/libpython2.7.so.1.0
#26 0x00007fe88408ec05 in __libc_start_main () from /lib64/libc.so.6
#27 0x000000000040071e in _start ()
===========================================================


Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib64/python2.7/site-packages/ROOT.py", line 669, in cleanup
    gROOT.EndOfProcessCleanups()
SystemError: void TROOT::EndOfProcessCleanups() =>
    problem in C++; program state has been reset
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib64/python2.7/site-packages/ROOT.py", line 669, in cleanup
    gROOT.EndOfProcessCleanups()
SystemError: void TROOT::EndOfProcessCleanups() =>
    problem in C++; program state has been reset

ROOT is 6.10/08
Python is Python 2.7.5

Steps to Reproduce (for bugs)

  1. From new cc7 machine
  2. Source gem-plotting-tools env
  3. Execute gemSCurveAnaToolkit.py -i listOfScanDates_Scurve.txt -c -s30 -v1 --anaType=scurveAna --drawLeg

Contents of listOfScanDates_Scurve.txt is reported below:

ChamberName	scandate
#GE11-VI-L-CERN-0002	current
#GE11-X-S-CERN-0001	current
GE11-X-S-CERN-0003	2018.01.17.14.29

Possible Solution (for bugs)

Probably something to do with how the TF1 object is stored/access in the TFile...in ROOT 5.34 it recognizes the TFile has been made with ROOT 6 and so just skips the TF1 (e.g. in ROOT 5.34 the class TF1Parameters does not exist).

However in ROOT 6 there's a crash.

Context (for feature requests)

Cannot investigate individual channel scurves and their fits.

Your Environment

  • Version used: Probably all of them
  • Shell used: zsh
@bdorney bdorney mentioned this issue Mar 12, 2018
Closed
@jsturdy
Copy link
Contributor

jsturdy commented Mar 12, 2018

Provide wgetable test data for test scripts to run in travis/gitlab-ci

@bdorney
Copy link
Contributor Author

bdorney commented Mar 13, 2018

Provide wgetable test data for test scripts to run in travis/gitlab-ci

You would like this uploaded to the repository under a .travis directory? If so which branch should the PR be made to? Maybe your private branch?

@jsturdy
Copy link
Contributor

jsturdy commented Mar 14, 2018 •
edited
Loading

No, I specifically don't want to put this test data in the repo, I just want a location that the CI can wget a file for testing from.
A possible alternative would be putting the test data in a dedicated docker container
The last ditch effort would be to put the data in the repo

Same for #62

@mexanick
Copy link
Contributor

mexanick commented Mar 14, 2018 via email

@bdorney
Copy link
Contributor Author

bdorney commented Mar 14, 2018

No, I specifically don't want to put this test data in the repo, I just want a location that the CI can wget a file for testing from.

I don't understand then, I thought wget requires a url? I could attach a test file to this issue page and then wget could use that? Is this the request?

A possible alternative would be putting the test data in a dedicated docker container

I don't know what a "docker container" is; is this something that is uploaded to travis? Sorry it's my ignorance here.

The last ditch effort would be to put the data in the repo

I think Misha's suggestion as having the files uploaded to a release is a good one, this then gives the user a test sample to play with if necessary.

@bdorney
Copy link
Contributor Author

bdorney commented Mar 14, 2018 •
edited
Loading

It might be worth pointing out that the gemSCurveAnaToolkit.py is also expecting the file format of which is identical to how $DATA_PATH is organized, as described in the documentation in the README.md here.

So in the example that is shown in the link above gemSCurveAnaToolkit.py is expecting a filestructure of the form:

$DATA_PATH/GE11-VI-L-CERN-0001/scurve/2017.08.11.16.30/ScurveData.root
$DATA_PATH/GE11-VI-L-CERN-0001/scurve/2017.08.14.20.54/ScurveData.root
$DATA_PATH/GE11-VI-L-CERN-0001/scurve/2017.08.30.15.03/ScurveData.root
$DATA_PATH/GE11-VI-L-CERN-0001/scurve/2017.08.30.21.39/ScurveData.root
$DATA_PATH/GE11-VI-L-CERN-0001/scurve/2017.08.31.08.28/ScurveData.root

What I would propose is that I provide the following:

  • listOfScanDates_Scurve.txt
  • tarball of SCurveData.root files in the appropriate filepath format

What may also be needed is a version of $GEM_PLOTTING_PROJECT/mapping/chamberInfo.py that has the following dictionaries compatible with the listOfScanDates_Scurve.txt specifically in this example it would be:

chamber_config = {
0:"GE11-VI-L-CERN-0001"
}

GEBtype = {
0:"long"
}

I can provide this to you and then you can choose how/where it should be hosted.

The tarball of ROOT files would also satisfy the request for data made in #62

@jsturdy
Copy link
Contributor

jsturdy commented Mar 14, 2018 •
edited
Loading

Another option is to add test data as attached file to the git release. Just my 2cents @mexanick

Well, this doesn't fit in the "run standard tests model that is the point of this addition.
No reason they couldn't also be attached to a release, providing a set of test data for a given release, but that's not the use case I'm imagining for this.
Here we'd just have a script that runs some unit tests, it needs some data, this data can easily be provided by some URL (or set up inside the container in some standard setup), and every PR would get tested against this.
If the data needs to change, the test script gets updated...

I can provide this to you and then you can choose how/where it should be hosted. @bdorney

OK, think about how the unit tests will need to be run and what would need to be included in the tarball, then I can take it from there

@bdorney bdorney mentioned this issue Mar 15, 2018
Merged
8 tasks
@AndrewLevin AndrewLevin mentioned this issue Jun 3, 2019
Merged
8 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jsturdy jsturdy

@bdorney bdorney

@mexanick mexanick

Labels
Priority: Medium Status: Help Wanted Type: Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jsturdy @bdorney @mexanick