say which file failed

[belforte]

when using numentries with a list of URL's I have hit this exception [1]
The lines before the stacktrace come from my script and say that it is trying to
get entries from all root files in
/eos/uscms/store/group/lpcbacon/15/JetHTRun2016H_07Aug17_v1_ddb8X
and there are 4308 of them.
I call root files all whose name end in .root and rely on uprood after that
(a usual situation, I am sure).

I 'd like the expection message to print the name of the bad file,
so that proper actions can be taken.

I cant' say if it is reproducible or if it was an xrootd server glitch.

[1]

JetHTRun2016H_07Aug17_v1_ddb8X
    # Files     :   4308
    Total Mbytes: 429454
    Avg.  Mbytes: 100
   Finding entries in 4308 root files ...

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-28225c5e62c4> in <module>()
    104         print (dSet)
    105         dsDir = baconprod_root + dSet
--> 106         baconInfo[dSet] = getFileInfoForDataset(dsDir)
    107         # save as pikle, will worry later about inderting in DBS
    108         with open(dSet+'.pkl','w') as fp:

<ipython-input-8-28225c5e62c4> in getFileInfoForDataset(dsDir)
     59         urllist = [fnaleos+path for path in dsLFNs]
     60         print ("   Finding entries in %d root files ..." % len(dsLFNs))
---> 61         nentriesGen = uproot.numentries(urllist, "Events", total=False, executor=executor)
     62         nentries = np.array(list(nentriesGen.values()))
     63         print("    Total entries: %d" % nentries.sum())

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/tree.pyc in numentries(path, treepath, total, localsource, xrootdsource, httpsource, executor, blocking)
   1618     else:
   1619         paths = [y for x in path for y in _filename_explode(x)]
-> 1620     return _numentries(paths, treepath, total, localsource, xrootdsource, httpsource, executor, blocking, [None] * len(paths))
   1621 
   1622 def _numentries(paths, treepath, total, localsource, xrootdsource, httpsource, executor, blocking, uuids):

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/tree.pyc in _numentries(paths, treepath, total, localsource, xrootdsource, httpsource, executor, blocking, uuids)
   1672 
   1673     if blocking:
-> 1674         return wait()
   1675     else:
   1676         return wait

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/tree.pyc in wait()
   1665     def wait():
   1666         for excinfo in excinfos:
-> 1667             _delayedraise(excinfo)
   1668         if total:
   1669             return sum(out)

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/tree.pyc in _delayedraise(excinfo)
     73         cls, err, trc = excinfo
     74         if sys.version_info[0] <= 2:
---> 75             exec("raise cls, err, trc")
     76         else:
     77             raise err.with_traceback(trc)

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/tree.pyc in fill(i)
   1637     def fill(i):
   1638         try:
-> 1639             file = uproot.rootio.open(paths[i], localsource=localsource, xrootdsource=xrootdsource, httpsource=httpsource, read_streamers=False)
   1640         except:
   1641             return sys.exc_info()

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/rootio.pyc in open(path, localsource, xrootdsource, httpsource, **options)
     75 
     76     elif _bytesid(parsed.scheme) == b"root":
---> 77         return xrootd(path, xrootdsource)
     78 
     79     elif _bytesid(parsed.scheme) == b"http" or _bytesid(parsed.scheme) == b"https":

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/rootio.pyc in xrootd(path, xrootdsource, **options)
     90     else:
     91         openfcn = xrootdsource
---> 92     return ROOTDirectory.read(openfcn(path), **options)
     93 
     94 def http(path, httpsource=HTTPSource.defaults, **options):

/uscms/homes/b/belforte/WORK/BD/Convert/CMSSW_9_4_13/python/uproot/rootio.pyc in read(source, *args, **options)
    133                 magic, fVersion = cursor.fields(source, ROOTDirectory._format1)
    134                 if magic != b"root":
--> 135                     raise ValueError("not a ROOT file (starts with {0} instead of 'root')".format(repr(magic)))
    136                 if fVersion < 1000000:
    137                     fBEGIN, fEND, fSeekFree, fNbytesFree, nfree, fNbytesName, fUnits, fCompress, fSeekInfo, fNbytesInfo, fUUID = cursor.fields(source, ROOTDirectory._format2_small)

ValueError: not a ROOT file (starts with '\x00\x00\x00\x00' instead of 'root')

[belforte]

ah.. the version:

uproot.__version__
'3.4.6'

[jpivarski]

That's a good point. I wonder if all error messages should say the name of the file. At least, all error messages that say there's something wrong with the file, as opposed to users doing something they shouldn't.

[belforte]

here the message is very clear as what's wrong in the file, my only wish is to find that file !
If I can throw the little I've learnt in the past: ideally error messages should give the information needed to reproduce the error in some simplified setup. Of course usually one pass a known file to an opener and there's not need to print the name, now you are so kind as to take a few K's of files in input !

[jpivarski]

Take a look at that diff: every error message that pertains to a file-formatting error now prints out the name of the file.

I hope I haven't added any errors-during-error-handling, since those can be frustrating and they're not well tested (error conditions generally don't get the coverage that features do). But it looks like I dotted all my 'i's.

[belforte]

after looking I can only be humbled by how quickly you get things done. I am of the kind "one line, two typos", do not count on me to find the t's you didn't cross :-) thanks.