Detector ID vs EXTNAME

[oczoske]

MICADO/FITS_extra_keywords.yaml constructs EXTNAME sequentially (at least that's the result of DET§.DATA, I am not familiar with the syntax), so that extension number i always has EXTNAME DET_i_.DATA. However, in MICADO's FPA_array_layout.dat, the detector id is not sequential, as number 4 has ID=6 and number 6 has ID=4. The EXTNAME should use the detector id to be consistent. I do not know how and if at all that is possible.

For a quick fix of the particular case of MICADO, I'm going to swap the rows in FPA_array_layout.dat to make ID sequential. I do not consider this as a solution to the issue, though.

[hugobuddel]

Perhaps it would be better to put the lines in the same order as the columns (/locations) in Table 4 of ELT_TRE_MCD_563000-0173_v0.5_ImagingSpectroRef.pdf , which I'm not sure they now are. See also #124.

About the detector IDs, I was under the impression that those are globally unique strings, and not integers, just like the filter IDs. That is, that they refer to a specific piece of hardware. For the filters this is useful because the physical filters might be replaced by a spare over the lifetime of the instrument, which will change the filter ID. The detectors will probably not be swapped (I suppose they can't), but I thought that these IDs are still similar.

I think these DETi.DATA are only there to associate different extensions with each other. E.g. there could hypothetically also be a DETi.ERROR or DETi.MASK extension that needs to be associated with the right DETi.DATA. But I don't think such extensions are relevant for either MICADO or METIS. (Euclid for example does deliver an 2-bit error layer with the raw data.)

For example, if say the middle detector breaks and cannot be used anymore, so there are only 8 functional detectors. Then it is my understanding that the detector IDs of the other 8 detectors will not change, but that the DETi.DATA will now go from 1 to 8 (and not from 1 to 9 while skipping 5).

I'm not entirely sure my understanding as described above is correct though. We need to think this through more. For METIS we will get raw data cubes with multiple exposures for each detector. So will there be multiple extensions with the same DETi.DATA? Or DETi's that go beyond the number of detectors? And I believe the order that the ICS writes the extensions is not fixed, and I don't know whether the order of the DETi in the file will always be 1-9 anyway (which I think is the case) or not.

FWIW, the "§" is there instead of "i" because I was looking for a unique placeholder that you could not use in either a key or a value of a FITS card. Using a non-ascii character was the safest choice in preventing silent errors, but it also causes loud errors in cases where no-one is interested in the headers, so maybe it was a bad choice.

[hugobuddel]

So to be clear, I think it is fine to put the order of the lines in FPA_array_layout.dat whatever is useful. Mainly because my position is that this order shouldn't matter at all :-).

However, it practice the order does matter. I understand that the previous order was chosen to make plotting easier (e.g. in the notebooks). So if we change the order in the file, then we should probably also update the notebooks to still plot the detectors in the most intuitive way. Perhaps by adding some helper function to ScopeSim that orders the detectors in a grid in an instrument-independent way, and call that function before plotting.

[hugobuddel]

From ESO-044156_7_DataInterfaceControlDocument.pdf :

EXTNAME and, if necessary, EXTVER and then EXTLEVEL, keywords are used to
distinguish different HDUs in the FITS file. It is strongly recommended that within
the same file, the values in all occurrences of EXTNAME (string) be unique and that
they uniquely describe the detector/chip/window combination used. If EXTNAME is
not unique then EXTVER (integer) keyword and – if not sufficient – also EXTLEVEL
(integer) keyword must be added to allow the combination of the keywords to
uniquely identify the HDUs within the file. It is specifically permitted to use these
keywords in the primary HDU.

With as example

EXTNAME = 'WIN1.CHIP1.OUT1'

See also VLT-SPE-ESO-19500-5667_DataFormat.pdf which describes how to have multiple extensions that should be associated with each other. They give as example a FITS file with

one extension with these headers

EXTNAME = 'IFU1.SCI'
ERRDATA = 'IFU1.ERR'
QUALDATA= 'IFU1.DQ '

and another with

EXTNAME = 'IFU1.ERR'
SCIDATA = 'IFU1.SCI'
QUALDATA= 'IFU1.DQ '

and then a thirnd with EXTNAME = IFU1.DQ `.

Following those examples it should probably have been DET1.SCI instead of DET1.DATA.

I'm still not sure what the convention for the number should be.