Update documentation for the 'Process image using a BioImage.IO model' Galaxy tool

[B0r1sD]

The wrapper for Process image using a BioImage.IO model should receive an update as it fails most of the time when actively trying to get it to run example models (@rabuono).

[B0r1sD]

Can @rabuono be added as collaborator to the repository where the issue resides? Thanks @beatrizserrano!

[kostrykin]

@B0r1sD Can you please add an example of the input for which the tools fails?

[B0r1sD]

@rabuono did all the testing, so he knows best.

[B0r1sD]

Link to original PR of the tool: bgruening/galaxytools#1391

[beatrizserrano]

Can @rabuono be added as collaborator to the repository where the issue resides? Thanks @beatrizserrano!

Added @rabuono! 🎉

[rabuono]

Challenge 1:
After testing, I cannot get 3D images to work.

Picking a model from the data library by choosing a "remote" file at "ML Models/bioimaging-models" and choosing a 2D compatible model like livecellsegmentationboundarymodel and running it on a 2D dataset, works.
It can run on the test dataset but also another dataset (as long as it is 2D) that I provide.

The same procedure, but choosing a 3D model to run on a 3D dataset (2 z slices) does not work.
Example of model tried: 3DUNetArabidopsisApicalStemCells

Challenge 2:
I cannot figure out which file format I should upload for the model.
The instructions say that a zip file can be used. Going to the https://bioimage.io/ and downloading a pytorch state dic and uploading said .zip file as the model does not work.
An example is livecellsegmentationboundarymodel, which works if pulled from uploading from "remote" file at "ML Models/bioimaging-models", but does not work if uploading the zip file from the model zoo. Is it the correct zip file that I am trying to update?

[kostrykin]

Still looking into Challenge 1, but we figured out Challenge 2:

Challenge 2: I cannot figure out which file format I should upload for the model. The instructions say that a zip file can be used. Going to the https://bioimage.io/ and downloading a pytorch state dic and uploading said .zip file as the model does not work. An example is livecellsegmentationboundarymodel, which works if pulled from uploading from "remote" file at "ML Models/bioimaging-models", but does not work if uploading the zip file from the model zoo. Is it the correct zip file that I am trying to update?

You need to download the Torchscript format file from the BMZ. This will give you a ZIP file, that you need to unzip. Inside the zip, you will find a .pt file. Galaxy will automatically recognize this is a ZIP file (no need to rename it or anything). This is the file that you need to feed into the tool.

It would be nice if this information could be added to the help text of the model input field of the tool!

[kostrykin]

Regarding Challenge 1, I managed to reproduce the error with a test case in bgruening/galaxytools#1544

[rabuono]

Regarding challenge 1,

Indeed, if I try 2D models and I provide my own 2d image (a simple 1024,512,1,1 8-bit tiff), it works.
I have tried the available 2D models present in "remote" file at "ML Models/bioimaging-models":
-livecellsegmentationboundarymodel
-CovidIFCellSegmentationBoundaryModel
-nucleisegmentationboundarymodel

Meaning that I can run the 2D models available on my own dataset and not only on the example dataset that the tool was made with.

[rabuono]

Regarding challenge 1,

If I try a same 3D image (256,256,2,1 8bit tiff) with the following 3D models available in "remote" file at "ML Models/bioimaging-models", it does not work.

• PlatynereisEMnucleiSegmentationBoundaryModel.pt
• PlatynereisEMcellsSegmentationBoundaryModel.pt
• HyLFM-Net-stat.pt
• 3DUNetArabidopsisApicalStemCells.pt
• HPANucleusSegmentation.pt failure might make sense as it expects 3 channel and single z image
• NeuronSegmentationInEM.pt
• HPACellSegmentationModel.pt failure might make sense as it expects 3 channel and single z image
• MitochondriaEMSegmentationBoundaryModel.pt

A few of the errors seem to be similar to RuntimeError: Given input size: (256x1x14x14). Calculated output size: (256x0x7x7). Output size is too small

[FynnBe]

One issue here might be that most models expect the first input tensor dimension to be a batch dimension...
If axes are known it's best to call bioimageio.core funcionality with bioimagieo.core.Sample objects comprised of bioimagieo.core.Tensor objects that have annotated dimensions (they are based on xarray.DataArray).

to address this it would be cool to inform the user of the required input axes and let them annotate the image dimensions accordingly... (or verify a guess).

Btw, HyLFM-Net-stat needs a fixed input size (b, 1, 1235, 1425) and processes light field microscopy images with 19x19 subimages to reconstruct volumetric images... in other words: it needs very specific input.

[rabuono]

Short update on challenge 1:
I made a 20 slice stack (256,256,20,1 8bit tiff)
I can run the two models on the BMZ "run model" tool:

• https://bioimage.io/#/?tags=cebrane&id=10.5281%2Fzenodo.7274275
• https://bioimage.io/#/?type=model&tags=3d,torchscript&id=10.5281%2Fzenodo.6334583

If I download the torchscript and upload the .pt to Galaxy, it still fails.

[rabuono]

Sorry for the accidental issue closing.

[kostrykin]

It would be very helpful if you could recreate all these failed cases as tests (or maybe not all, but a representative subset), similar to what I have done here: bgruening/galaxytools#1544

This will greatly facilitate fixing the issue in the future.