feat : add PU files

[Ming-Yan]

run3 changes

• add pu files, adapt correction will available dicts
• add Muon_RunDv1 to data json files
• improve file splitting for dask/lxplus

Other

• remove redudant puwei file
• python3.7->3.10 in pipeline
• add autorebin for plotting tools

[demuller]

Hi @Ming-Yan the PR looks mostly fine. I have mainly comments on the README, one question about the logic used in the job splitting using the indices in runner.py, and spotted an issue with the CI pipelines for python 3.10

[demuller]

I am not sure if this else (and the if directly above) is correct...

For the simple case that we have only one sample with 150 files, i.e. 3 batches of 50 files [0:50,50:100,100:150]:

If I only managed to finish the first job (i.e. the one with findex = 0), and want to restart the remaining ones with --index (0,1). For findex == 1, it correctly sets mins to 50, but then after incrementing findexby 1 in L 668, it would set mins to 0 instead of 100 because of the if condition in L 659, or do I misunderstand something here?

[Ming-Yan]

These conditions are set outside the while loop, which means when the findex increased, the mins would be automatically update with maxs as what written in the while loop

if int(args.index.split(",")[1]) == findex:
    mins = findex * 50
else:
   mins = 0

The conditions above did not check after entering the while loop.

[demuller]

Ah right, I have missed that part

[Ming-Yan]

I play around the bottom scripts but not seeing any suspicious thing there

import json
with open("metadata/data_Winter22_mu_BTV_Run3_2022_Comm_v1.json") as f:
    sample_dict = json.load(f)

index="1,3"
findex = int(index.split(",")[1])

for sindex,sample in enumerate(sample_dict.keys()):

    if sindex<int(index.split(",")[0]):continue
    if  int(index.split(",")[1])==findex:mins=findex*100
    else:mins=0
    while mins < len(sample_dict[sample]):
        splitted = {}
        maxs = mins + 100
        splitted[sample] = sample_dict[sample][mins:maxs]
        mins = maxs
        findex=findex+1
        print(sample,splitted)