Unable to finish CLI processing for certain images

[sarthakpati]

Hi @nathanhillyer,

I am trying to extract features for an image/mask combination as follows:

inputMask.nii.gz
inputImage.nii.gz

But the following command did not work:

collage_cli + " -i " + inputImage + " -m " + inputMask + " -s 1 -b 20 -o " + outputCSV

With the following output:
collage_output.log

And this is the traceback:

Traceback (most recent call last):
  File "collageradiomicscli.py", line 93, in <module>
  File "click\core.py", line 829, in __call__
  File "click\core.py", line 782, in main
  File "click\core.py", line 1066, in invoke
  File "click\core.py", line 610, in invoke
  File "collageradiomicscli.py", line 44, in run
  File "collageradiomics.py", line 602, in execute
ValueError: could not broadcast input array from shape (3,3,5,13,2) into shape (5,5,5,13,2)

However, full-size images work fine. Could you please help? I am guessing this might be as simple as including some error-checking on my side.

Cheers,
Sarthak

[nathanhillyer]

@sarthakpati Thanks for the report and attaching the image + mask, looking into this.

[sarthakpati]

@nathanhillyer: any update on this?

[nathanhillyer]

@sarthakpati I mistakenly thought this was resolved when I saw CBICA/CaPTk#1288 merged.

My fault, we're looking into this now.

[nathanhillyer]

I think the problem here is the mask is too big. It encompasses the whole image and our dominant angle calculation is not accurate at the edges. So the mask should be within the image minus svd_radius*2+1 (diameter).

I think adding error checking on both yours and our end will resolve this.

But I'm going to sync up with the team to make sure I'm not missing something here. Thanks for your patience!

[sarthakpati]

I think the problem here is the mask is too big. It encompasses the whole image

We will have a lot of cases where the mask will span the entire image [ref]. In fact, I am seeing similar behavior for exceedingly small masks as well (in the order of a few voxels in an image of 240x240x155).

But I'm going to sync up with the team to make sure I'm not missing something here.

Thanks!

[nathanhillyer]

We can reduce the mask size automatically in the case where it spans the whole image and still have the statistics compute.

We are also going to implement a minimum size for the images. Pallavi and Satish highly recommend leaving the svd radius as 5, which would mean the image would ideally be 50x50 minimum in the x, y dimensions. We'll talk with them this week and see how they want to handle small images.

[sarthakpati]

Cool. It would be great to not give a compute failure for smaller images and make adjustments internally. Let me know how I can help!

[nathanhillyer]

@sarthakpati I talked with Pallavi & Satish and we're going to make a minimum size for images going through collage. We can't in good faith compute statistics for images under 50x50 and have dependable results. We'll update the error message accordingly, and suggest leaving collage radiomic features out of smaller images.

They also highly discourage running a mask that is the entire image size. Our algorithm is computationally expensive and doesn't scale well to full images. It's much better to target a certain area or areas. As it looks like your image pipeline is setup for full images in some cases already, we suggest leaving collage out for those cases, as it's not the intended use case. We ruled out supporting a full image mask because we don't want to encourage such usage.

We can make helper methods (if we don't have them already) where you can pass a minX, maxX, minY, maxY, and have collage run over a passed in smaller region if you want these full images to be ran. Would that work?

[sarthakpati]

We can't in good faith compute statistics for images under 50x50 and have dependable results. We'll update the error message accordingly, and suggest leaving collage radiomic features out of smaller images.

Completely fair.

They also highly discourage running a mask that is the entire image size

We cannot control the mask that the user is providing, to be honest. If they choose to provide a full image mask, we are obliged to do the computation. Additionally, the lattice computation (the part of the pipeline where we are seeing this particular behavior) is the one where we are splitting the main input image into a bunch of smaller images (which consequently can have the full mask, depending on where they are sourced from).

We can make helper methods

Thank you, but I doubt we will be able to do much with them. I will simply disable Collage processing for smaller images. Is there a minimum size that I should be looking at?

[satishev]

We cannot control the mask that the user is providing, to be honest. If they choose to provide a full image mask, we are obliged to do the computation. Additionally, the lattice computation (the part of the pipeline where we are seeing this particular behavior) is the one where we are splitting the main input image into a bunch of smaller images (which consequently can have the full mask, depending on where they are sourced from).

If a bigger image is being split into a bunch of smaller images, and the size of each of the smaller images is not huge (think in the order of typical tumor sizes), COLLAGE will run in a reasonable amount of time. The problem is if one tries to run COLLAGE on an entire 256 x 256 (or larger) image in one shot, computing gradient co-occurrences etc will take way too long.. since these are done within sliding window neighborhoods. Does that make sense?

Thank you, but I doubt we will be able to do much with them. I will simply disable Collage processing for smaller images. Is there a minimum size that I should be looking at?

Like Nathan mentioned, I suspect 50 x 50 is a reasonable minimum. For reference, we have typically run COLLAGE on ROIs that are the size of a tumor + peritumor annotation where it executes in a reasonable amount of time (trying to give you a sense of the ROI sizes that we have been working with).

[sarthakpati]

We cannot control the mask that the user is providing, to be honest. If they choose to provide a full image mask, we are obliged to do the computation. Additionally, the lattice computation (the part of the pipeline where we are seeing this particular behavior) is the one where we are splitting the main input image into a bunch of smaller images (which consequently can have the full mask, depending on where they are sourced from).

If a bigger image is being split into a bunch of smaller images, and the size of each of the smaller images is not huge (think in the order of typical tumor sizes), COLLAGE will run in a reasonable amount of time. The problem is if one tries to run COLLAGE on an entire 256 x 256 (or larger) image in one shot, computing gradient co-occurrences etc will take way too long.. since these are done within sliding window neighborhoods. Does that make sense?

Absolutely, just that we have no control over how a user will initialize this and would want to run their experiment. The same holds true for other texture-based features getting extracted using CaPTk, BTW.

Thank you, but I doubt we will be able to do much with them. I will simply disable Collage processing for smaller images. Is there a minimum size that I should be looking at?

Like Nathan mentioned, I suspect 50 x 50 is a reasonable minimum. For reference, we have typically run COLLAGE on ROIs that are the size of a tumor + peritumor annotation where it executes in a reasonable amount of time (trying to give you a sense of the ROI sizes that we have been working with).

I can set that as the minimum for COLLAGE computation on my end.

[satishev]

@nathanhillyer based on what Sarthak is saying, I am thinking we should only specify a minimum size for running COLLAGE (since that is based on real constraints of what the feature means). If the mask is the size of the entire image, we should probably just pop up a strident warning and let the end-user deal with the consequences of a looooong execution time.

[nathanhillyer]

@satishev Okay, works for me! We'll update to 50x50 as the minimum size, and make some tweaks that allow a fully masked image to be computed.

[nathanhillyer]

@sarthakpati These two issues are resolved in version 0.3.0 which will be on PyPi shortly.