handle out of plane segmentation better

[pieper]

Request

These kidney and tumor segmentations in TCIA should be displayable in IDC:
https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=61081171

Here's an example of one in OHIF sandbox:

https://idc-sandbox-000.firebaseapp.com/projects/idc-sandbox-000/locations/us-central1/datasets/sdp-dev/dicomStores/KiTS/study/1.3.6.1.4.1.14519.5.2.1.6919.4624.208465220391024702985828804865

They are not currently displayed because the SEG are coronal but the CT is axial, leading to this message in the console:

This segmentation object is not in-plane with the source data. Bailing out of IO. It'd be better to render this with vtkjs.

What feature or change would you like to see made?

Would be better to give a user-level dialog explaining why the seg isn't displayed.

Maybe better offer to reformat the seg in the plane of the image.

Why should we prioritize this feature?

It's a fairly common use case (or at least it comes up often enough.

[pieper]

Here's an example of what one of these datasets should look like:

[JamesAPetts]

Its fun to see my "TODO" warnings from the last project week in Gran Canaria getting hit haha.

Maybe better offer to reformat the seg in the plane of the image.

That is interesting, I originally thought we'd just render it "as is" with vtkjs, but that might cause more problems than it solves.

Right now we load these SEGs into a labelmap in cornerstone, and vtkjs piggy backs off this whilst rendering. This has the benefit of preventing posible duplication and makes book keeping easier.

We could either resample as you suggest and keep doing this, or we can add another route and display the SEG as is in vtkjs. The latter sounds cleaner, but it'd be annoying to limit the user to not see the segs in cornerstone.

Any idea how we could easily convery this to the user @dannyrb ?

[pieper]

Displaying off-axis seg only in vtk and not in cornerstone seems like an okay workaround from my perspective, at least for now. Resampling is also okay, but since it's lossy in general It's not my favorite option.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[JamesAPetts]

SEG are coronal but the CT is axial

I must of originally glazed over this, if its not oblique we can "just" rotate this and get it in cornerstone and everything else should just work (i.e. 2D and 3D). For obliques we will really have to just display them out of plane. I'll try and get this to work.

[JamesAPetts]

Moving this back to Candidate:

• We now show messages that oblique + orthogonal-but-not-in-acquisition-direction SEGs can't be loaded.

• We would like to actually deal with the above cases, but its not important for the IDC MVP.

[pieper]

this is definitely on the right track - but there seems to be an infinite loop of error boxes showing up when I load the study. Do you get the same thing?

[pieper]

This should be addressed as part of the update to the core off screen rendering functionality. Related to #2238

[sedghi]

@igoroctaviano @fedorov Do you think this is still valid?

[fedorov]

No, I think this can be closed.