Add support for GE files with an EDF header

[psavery]

This file type started coming from APS in ~2022 with a .edf.ge5 extension.

In fabio, these were being interpreted as EDF files because of their EDF-like header. With the changes in this PR, the file is interpreted as a GE file instead and the frames are read in correctly.

The added logic is as follows:

1. If the file is identified as EDF, we check the file extension, and if the file extension is ge plus a number, it is interpreted as GE instead.
2. We check for EDF header formats in the GE reader, and if we have a match, we read in the header using EDF style and grab the number of frames from there.

I have an ~80 MB dark example... let me see if we can make that public for testing.

Fixes: #553

[psavery]

I can share the dark file, which is ~80 MB! Is that too large for public testing, or is it okay?

[jonwright]

Hi Patrick - I would suggest to gzip the file to make it smaller for testing. Normally fabio reads that transparently, but it breaks the format guessing from file extensions....

[kif]

Hi, in FabIO, files are distributed as bz2 format and then decompressed and recompressed in gzip. This saves a lot of disk space. Don't worry too much about the size of that file.

Please send me such a file to put it on edna (our server, used for distributing those images).

Then remains the non regression test to implement.

[jonwright]

For now it just misses the testcase ... ?

[psavery]

I sent a test case by email. We can make that public.

[psavery]

I added the file to the GE tests! Let me know if anything further is needed for merging this PR.

[jonwright]

I had a quick look. and it seems like compression gets broken like this, as openimage is not picking up the filetype from the name. Historically, I thought edfimage used to complain when the file is the wrong size?

In [1]: import fabio, fabio.test.utilstest

In [2]: fabio.open( fabio.test.utilstest.UtilsTest.getimage("dark_before_000428.edf.ge5") ).nframes
Out[2]: 10

In [3]: fabio.open( fabio.test.utilstest.UtilsTest.getimage("dark_before_000428.edf.ge5.gz") ).nframes
Out[3]: 1

In [4]: fabio.open( fabio.test.utilstest.UtilsTest.getimage("dark_before_000428.edf.ge5.bz2") ).nframes
Out[4]: 1

[jonwright]

So I guess the question is whether anyone would want to support reading dark_before_000428.edf.ge5.bz2, I will ping the private email thread

[psavery]

We can potentially modify the regex to allow .ge5.* as well.

Also, in this PR, I am assuming that Num_Images will be in the EDF header, since it was there in our example files. I wonder if it's safer to just figure out the number of frames by the file size instead? But I guess we could always change that in the future too, if needed.

[psavery]

fabio.open( fabio.test.utilstest.UtilsTest.getimage("dark_before_000428.edf.ge5.bz2") ).nframes

I just fixed this.

[jonwright]

Looks good to me

[psavery]

Thanks @jonwright and @kif!

I guess we'll need to wait for a new release on pypi to get access to these features in a release?

[kif]

indeed ... I am making 2 to 3 releases a year.
The support for numpy2 is something important, and once h5py is compatible it would make sense to have such a release.