-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Image labeling at whole-well level #104
Comments
As of 873e854, we have a first naive implementation, where the mask is explicitly upscaled before being stored on disk (see fractal-analytics-platform/fractal-tasks-core#24 for a discussion). This allowed for a quick implementation and visualization of results. See for instance this result for a 2x2 well segmented at level 1: Next steps:
Notes
|
This segmentation result looks great @tcompa ! 👏🏻
With our current approach being wells always being saved as single FOVs, we don't need to go into this now. But worth considering down the road :) Very much looking forward to seeing this run on whole wells through Fractal. I will be preparing additional test sets where we want to test 2D segmentation of wells on organoids (much larger structures), which should work by changing some of the cellpose parameters :) |
As a future reference, segmenting an array of shape Traceback (most recent call last):
File "../../image_labeling_whole_well.py", line 210, in <module>
image_labeling_whole_well(
File "../../image_labeling_whole_well.py", line 101, in image_labeling_whole_well
mask, flows, styles, diams = model.eval(
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/models.py", line 227, in eval
masks, flows, styles = self.cp.eval(x,
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/models.py", line 536, in eval
masks, styles, dP, cellprob, p = self._run_cp(x,
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/models.py", line 636, in _run_cp
outputs = dynamics.compute_masks(dP[:,i], cellprob[i], niter=niter, cellprob_threshold=cellprob_threshold,
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/dynamics.py", line 725, in compute_masks
mask = remove_bad_flow_masks(mask, dP, threshold=flow_threshold, use_gpu=use_gpu, device=device)
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/dynamics.py", line 580, in remove_bad_flow_masks
merrors, _ = metrics.flow_error(masks, flows, use_gpu, device)
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/metrics.py", line 282, in flow_error
dP_masks = dynamics.masks_to_flows(maski, use_gpu=use_gpu, device=device)
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/dynamics.py", line 277, in masks_to_flows
mu, mu_c = masks_to_flows_device(masks, device=device)
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/dynamics.py", line 152, in masks_to_flows_gpu
mu = _extend_centers_gpu(neighbors, centers, isneighbor, Ly, Lx,
File "/data/homes/fractal/.conda/envs/fractal/lib/python3.8/site-packages/cellpose/dynamics.py", line 83, in _extend_centers_gpu
grads = T[:, pt[[2,1,4,3],:,0], pt[[2,1,4,3],:,1]]
RuntimeError: CUDA out of memory. Tried to allocate 1.29 GiB (GPU 0; 14.76 GiB total capacity; 12.42 GiB already allocated; 561.44 MiB free; 13.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF Now trying with level 2. |
Labeling a single Z plane of a 9x8 well at level 2 (corresponding to a
The memory peak for this run is ~7 G, and the runtime is ~20 seconds. Labeling options are probably somewhat off, since it seems that we find less labels than expected: |
BUGFIX: In the previous comment I kept And the labels look much more reasonable: |
@tcompa This looks great! :)
This is a good proxy given the input 40. May even be an interesting setup for the users afterwards: We take the diameter at full res and calculate a relevant diameter parameter for the pyramid level |
As a future reference, when labeling (e.g.) level 2 (and with "datasets": [
{
"path": "0",
"coordinateTransformations": [{
"type": "scale",
"scale": [1.0, 4.0, 4.0]
}]
},
...
], I'm still trying to understand what This is hard to understand ex-post, because it seems that napari doesn't load pyramids when showing a single FOV - and I cannot show a well/plate, since it seems that they do not support labels (*). Thus I'm only viewing level 0. @jluethi, is this question possibly something obvious to you? (*) See ongoing work on ome/ome-zarr-py#207 and ome/ome-zarr-py#65. |
As of 136931e, we removed upscaling and we added the correct Tests with napari confirm that the scale is set correctly (without any explicit array upscaling). We still have to set the correct |
Yes, the coordinate Transformation should contain this scale. It should actually be relative to whatever we have in the main zarr file. We started this discussion when parsing Yokogawa metadata (see here: fractal-analytics-platform/fractal-tasks-core#25). The UZH test datasets we are using have [1.0, 0.1625, 0.1625] at level 0 and higher levels once we go up (see here: fractal-analytics-platform/fractal-tasks-core#25 for some actual values with 3x coarsening). We haven't yet included them in the parsing, but that should come soon with metadata parsing and is required to use ROIs well (fractal-analytics-platform/fractal-tasks-core#24). We will want to use this scale metadata per layer to calculate physical pixel dimensions for a given absolute spatial position at a given pyramid layer. Conclusions:
|
Thanks for the explanation/reminder. This is now fixed with 20371d2, to reproduce this behavior:
As for the more general expected behavior (where |
At this point, the prototype task is ready and what is missing is only its integration within fractal. |
Great! Looking forward to testing this in Fractal and applying a few new test sets to it :) Let's make sure we handle the general scales when doing the ROIs then and then update this task to also handle the scales. But first, we can test the prototype task in Fractal on original FOVs and on whole wells in 2D :) |
This is now ready for tests: #109 |
With the successful tests in #111, I think we can close this issue. We can later refactor things a bit to support arbitrary pyramid levels as an input when we add ROI support to the labeling task 🙂 |
As of #64 (comment), planned use cases for image labeling are
Use case 1 is discussed in #64.
This issue concerns use case 2, namely the possibility to segment a whole 2D well.
The text was updated successfully, but these errors were encountered: