Mosaic dimension in specification

[yfukai]

Hi, I'm working with tiled images with the "mosaic" dimension (corresponding to the stage position of a tiled large image without stitching). As far as I see from the current specs (Editor’s Draft, 31 January 2023), the only way to explicitly store those images is to use the transitional 3.2. "bioformats2raw.layout". I'm wondering if I'm getting this point correct, and in that case, if something corresponding to this dimension will be included in the specification in near future. Thanks!

[will-moore]

If I understand correctly, you wish to store a collection of Images, and you want to specify the position of each?

You can use bioformats2raw.layout to group the images in the collection, then you can specify a translation within the multiscales coordinateTransformations (https://ngff.openmicroscopy.org/latest/#trafo-md) of each Image.

NB: translation within coordinateTransformations on multiscales (v0.4) is not supported by current tools: https://ome.github.io/ome-ngff-tools/

An alternative if you simply wish to layout the images in a grid of rows and columns is to use the NGFF "plate" layout: https://ngff.openmicroscopy.org/latest/#plate-md for High-Content-Screening, assigning each image to a Well.

[yfukai]

Hi @will-moore, thanks for your response! Yes, I'm trying to store a collection of images with, for example, tiled positions (something like Fig. (a) of this ).

I think It is common practice to take a "tiled" image with a grid position list. For example, the aicsimageio package provides the "mosaic" dimension as well as a function to get their positions. Please note this is given independently to another index, scene, which represents, for example, the plate well positions. I was wondering if there's any way to represent something similar for OME-NGFF. If there is native support for this, it would be much easier to process (for example, stitch) and visualize (for example, via napari) such tiled images.

bioformats2raw.layout sounds OK, but it's transitional, and I was wondering if there's any plan to implement this as a bit more concrete spec.

NGFF "plate" layout

Thanks for your suggestion, but I rather feel I should distinguish the "plate" positions from the mosaic position, as they have different requirements. (We usually don't care about the relative positions of different plate wells, but we do so for mosaic positions. In that sense, I believe the "plate" position corresponds to scene for the aicsimageio image model.)

[will-moore]

The mosaic position could be handled with "translation within coordinateTransformations on datasets (v0.4)" - see the samples with that feature in the table at https://ome.github.io/ome-ngff-tools/ - which also shows that napari is the only tool that currently supports that metadata.

You can test it in napari (with napari-ome-zarr plugin installed)

Open the parent image with command line...

$ napari https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457227.zarr

Then in the napari console, open the image that has a "translation" applied:

viewer.open("https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457537.zarr", plugin="napari-ome-zarr")

With some playing of Z/T sliders and contrast levels, you can see that the smaller 2nd image has an X/Y offset compared to the first:

By opening the 2nd image in ome-ngff-validator: https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0101A/13457537.zarr
you can see that the dataset.coordinateTransformations has a scale (required) from pixels to microns, then a translation in microns:

According to the spec, each dataset should have it's own coordinateTransformations.scale and optional translation but currently napari only reads coordinateTransformations from the first dataset.

So, you can use this to specify translation of each image.
However, napari won't currently open a whole collection of images under bioformats2raw.layout container, so the available tools won't support your use-case currently, as far as I'm aware, but the Spec does.

Another option is to simply "stitch" the mosaics into a single larger image. This approach has been adopted by some users for grouping fields within an HCS Well, as it reduces the number of separate images that need to be loaded when viewing a whole Plate: see #137 for discussion.
It will depend on your use-case whether you prefer to keep the images separate, but the Zarr format should handle 'sparse' data quite well (omitting chunks and/or compressing blank areas).

[yfukai]

Thanks @will-moore for your detailed answer and instruction! I tried the example you provided and confirmed the coordinateTransformations works as expected. I also understood to what extent the
the specification is supported by napari.

I'm developing software for shading correction (https://github.com/peng-lab/BaSiCPy) and stitching (https://github.com/yfukai/m2stitch), and my use case is to save (and possibly visualize) shading-effect-corrected but non-stitched images with multiple fields of view (FoV, mosaic position). To prepare a standardized input/output format for those tools, I'm still wondering about the following.

Currently, there are two expected layouts:

• "Images" layout for a single FoV
• "High-content screening" layout for multiple wells, each including multiple FoVs.

Am I correctly understanding that the current transient bioformats2raw.layout is between those layouts, representing multiple FoVs but a single scene? If so, it would be great if this "multiple FoVs but a single scene" scenario is explicitly included as a concrete specification in future.

[will-moore]

There's an issue for that at ome/napari-ome-zarr#71

[yfukai]

Oh, I missed it; thanks, @will-moore! It will be awesome if this scenario (unstitched single well with multiple FoVs) is explicitly included in the spec and implementations. I believe it is common practice to store and visualize non-stitched images, as shading correction and stitching are independent steps.

[melonora]

I am working on an initial proposal to be included in the ome-ngff spec for this. I can tag you when ready for feedback:)

[melonora]

From my understanding is that the current coordinate transform with translation cannot handle the overlap that exists between FOVs when dealing with images that need to be stitched. As in both tiles would be shown in the overlapping region when applying the translation for stitching. Am I correct in my understanding?

@will-moore Is there currently already work ongoing to support this? @joshmoore mentioned to me that @bogovicj was working on this. Do you know where it would be possible to read up on this?

[will-moore]

"both tiles would be shown in the overlapping region" - I'm not sure if napari supports the blending of overlapping tiles you're looking for? If you try the various blending options for each layer, can you achieve the effect you want?
When a reader passes multiple Image Layers to napari, it's possible to set the blending option on each.

[melonora]

This is not solely a viewing problem, but also an analysis problem. For example, when doing segmentation either you would deal images in which the overlapping regions of the tiles have double or quadruple the intensity or you would do segmentation per tile and identify certain objects multiple times after which you would have to merge those.

I think ideally the spec should support an offset indicating from where onwards a tile should be read. Let's say we have tile 1 (left) and 2 (right) with dimension y and x being 500 pixels and they overlap horizontally 250 pixels. The spec could indicate that tile 1 needs to be read entirely, while tile 2 would be sliced -> tile2[ : , 250:]. What do you think about this?

[will-moore]

That sounds quite complex! I think this could be handled by a "crop" transformation. There is some mention of crop in a current proposal for additional transformations: https://github.com/ome/ngff/pull/138/files (see the index.bs diff) - (I think this is what Josh was referring to), but I'm not sure that a "crop" transformation is being proposed there at this stage.

[melonora]

I just went over it. I do see that crop are saved separately from the actual volume. Personally, I would not be in favor of this as it would store most of the data twice. Something that is quite costly when dealing with high volume multiplexed imaging. It is also not necessarily a crop in the sense of a region of interest.

It is indeed complex, but I do think there is a use case for people working with microscopes that do not apply a stitching on the fly or in case they would like to see for themselves whether stitching is working correctly and otherwise change the stitching algorithm.