First test use in early May

[steinnymir]

In early may, there will be beamtime at DESY where we would like to implement the event-resolved data processing.

Therefore, we should get a running version of the SED processor, together with the readout from the LAB format (@zainsohail04 you know what I mean), and output functions to save the data as .tiff files.

[rettigl]

What exactly is required for this? Should we define a concrete target for this?

[daviddoji]

That's the main question to answer. Otherwise, we will start implementing "secondary" and "nice-to-have" things :)

[steinnymir]

Basically, we need to complete the Setup Basics milestone.

[steinnymir]

In short, we will need to read the data from an hdf5 format containing lists of events and other parameters, binn it and save it as .hdf5 and/or .tiff.

The first part does not belong to this package. The current readout implementation in hextof-processor can be used.

Requirements

• importing the dataframe
• binning (done)
• save the binned data as hdf5 and tiff
• simple API, to transform the data in just a few simple lines of python code

Optional extras

• calibration methods
  • energy
  • momentum
• filtering dataframe
• call pipeline from command line
• metadata handling system

These are the things that come to mind so far. They don't need to be refined in the final version, but need to work enough to test the new acquisition system.
In any case we should schedule a meeting soon and discuss the status of the project as well as short and long term goals. I'll send around an email for that.

[daviddoji]

@steinnymir @zainsohail04 do you have h5 files, with the structure that the ones you're going to produce in May, to play with?
I'd like to test an implementation that does not produce those intermediate parquet files.

[steinnymir]

Some data is available on asap3. You will need permissions to access the beamtime directory. I will ask Dima if he can give them to you.
This is not exactly what we will have, but its as close as it gets. We are still waiting on news about the final version.

[rettigl]

I think part of that first milestone should be to include caller functions for all functionalities in the wrapper class and produce a demonstrator notebook for this. With the existing functionality (binning, saving, loading), we could already work on this.

[steinnymir]

I agree. Could you make an issue for this an assign it to the milestone?