Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extensions of the surface-based metric #4103

Closed
wyli opened this issue Apr 11, 2022 · 9 comments · Fixed by #6681
Closed

extensions of the surface-based metric #4103

wyli opened this issue Apr 11, 2022 · 9 comments · Fixed by #6681
Labels

Comments

@wyli
Copy link
Contributor

wyli commented Apr 11, 2022

Is your feature request related to a problem? Please describe.
Follow up of #4050,
this is a feature request to extend the surface-based metric module for 3D, with the support of more detailed quantizations when computing local surface lengths/areas (such as the computation proposed in https://github.com/deepmind/surface-distance).

#4050 (comment)

the way 3D surfaces & surface distances are handled in deep mind is different from MONAI: Instead of defining surfaces at surface voxel boundaries, surface points are located on a grid that is shifted by half of the grid spacing along each axis (cf. Fig. 10 in their paper):

grafik

I think it should be discussed more broadly whether to keep the surface computation currently used in MONAI, or to switch to the deep mind computation, or to establish a switch such that the user can decide on the computation. This choice should be made consistently for all boundary-overlap metrics.

cc @SilviaSeidlitz

@wyli wyli added Module: metrics metric for model quality assessments Contribution wanted Feature request labels Apr 11, 2022
@wyli wyli changed the title extensions of surface Dice metric extensions of the surface-based metric Apr 11, 2022
@jakubMitura14
Copy link

Hausdorff distance and Average surface distance in Monai are already implemented using the deep mind library you are referring to, what exactly is not present that you need?

You can also lok into some fast approximation that I am now developing in

#4078

this can give potentially the performant approximations of the quantities you are requesting

@wyli
Copy link
Contributor Author

wyli commented Apr 13, 2022

what exactly is not present that you need?

please see #4050 (comment)

@jakubMitura14
Copy link

jakubMitura14 commented Apr 13, 2022

ok now I get it, I will consider wheather I can contribute to it !

@hreso110100
Copy link

Hi can Average surface distance be considered same as Average Hausdorff distance ? Because I cant see Average Hausdorff distance implementation in monai. Thanks for reply

@jakubMitura14
Copy link

Hello I can not say wheather they are mathemathically the same as my mathemathical skills are not sufficient - I suppose not exactly, but I ran some statistical experiments suggesting that they are closely correlated - correlation around 0.9 and very low p values.

@hreso110100
Copy link

Maybe @wyli can confirm this ?

@wyli
Copy link
Contributor Author

wyli commented Jun 23, 2022

no they are different distances @hreso110100

@hreso110100
Copy link

And is there some plan to add also this metric ?

@jakubMitura14
Copy link

Here is working Morphological algorithm (pytorch extension - not MONAI)
https://github.com/jakubMitura14/Hausdorff_morphological

Hovewer this is fast morphological approximation - quite accurate and having very good performance, still not exact, also it can be considered rather research than production software

To be honest recently I got a new job and was unable to spend time debugging MONAI pull request here is the pull request to MONAI - it is not working, would you like to contribute in debugging :)
#4078

wyli pushed a commit that referenced this issue May 24, 2023
### Description

This PR extends the `SurfaceDiceMetric` for 3D images. The
implementation already uses generic functions to obtain the boundary
edges and compute the distance, the extension just
- removes the assertion that the input is 2D (`[B,C,W,H]`).
- updates the docstrings
- adds a test case in
`TestAllSurfaceDiceMetrics.test_tolerance_euclidean_distance_3d`

fixes #5906
mentioning #4103

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [x] New tests added to cover the changes.
- [x] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [x] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [x] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Bryn Lloyd <lloyd@itis.swiss>
@wyli wyli closed this as completed in #6681 Jul 5, 2023
wyli added a commit to wyli/MONAI that referenced this issue Jul 5, 2023
Fixes Project-MONAI#4103

### Description
- considers spacing and subvoxel borders when computing surface Dice
- reimplemented
http://medicaldecathlon.com/files/Surface_distance_based_measures.ipynb
using pytorch API
- there's possibility of speed up once
Project-MONAI#1332 is addressed


### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [x] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [x] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [x] In-line docstrings updated.
- [x] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants