Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FROC metrics for 3D data #5172

Closed
akiliyiu opened this issue Sep 19, 2022 · 2 comments · Fixed by #6528
Closed

FROC metrics for 3D data #5172

akiliyiu opened this issue Sep 19, 2022 · 2 comments · Fixed by #6528

Comments

@akiliyiu
Copy link

Is your feature request related to a problem? Please describe.
To be used on 3D data such as MRI, CT images

Additional context
Notice that the current froc metrics only support 2d data, wonder if it is possible to extend it to 3d data? Thanks

@kretes
Copy link
Contributor

kretes commented May 17, 2023

The only method that needs to be adapted is:

def compute_fp_tp_probs(
    probs: NdarrayOrTensor,
    y_coord: NdarrayOrTensor,
    x_coord: NdarrayOrTensor,
    evaluation_mask: NdarrayOrTensor,
    labels_to_exclude: list | None = None,
    resolution_level: int = 0,
) -> tuple[NdarrayOrTensor, NdarrayOrTensor, int]:

from
https://github.com/Project-MONAI/MONAI/blob/8e725aa2f3c50ffd1c0c45e8917d85f20c6637d0/monai/metrics/froc.py#LL22C1-L81C1

And it seems impossible to introduce 3D support without breaking the API.

I could submit a PR if we agree on an API. E.g.:

def compute_fp_tp_probs(
    probs: NdarrayOrTensor,
    coords: NdarrayOrTensor,    
    evaluation_mask: NdarrayOrTensor,
    labels_to_exclude: list | None = None,
    resolution_level: int = 0,
) -> tuple[NdarrayOrTensor, NdarrayOrTensor, int]:
    """
    Args:
        probs: an array with shape (n,) that represents the probabilities of the detections.
            Where, n is the number of predicted detections.
        coords: an array with shape (n, n_dim) that represents the coordinates of the detections in the same order as in `evaluation_mask`.        
        evaluation_mask: the ground truth mask for evaluation (need to have same n_dim dimensions).
        labels_to_exclude: labels in this list will not be counted for metric calculation.
        resolution_level: the level at which the evaluation mask is made.
     """

this one would support arbitrary dimensions

@wyli
Copy link
Contributor

wyli commented May 17, 2023

thanks, for backward compatibility, we can create compute_fp_tp_nd and then reimplement compute_fp_tp_probs using compute_fp_tp_nd. also I think resolution_level should be remain in compute_fp_tp_probs as that's not a generic ND concept.

@kretes kretes mentioned this issue May 18, 2023
6 tasks
wyli pushed a commit that referenced this issue May 24, 2023
Fixes #5172 .

### Description

Implementation of FROC metric in ND

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [x] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [X] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [X] In-line docstrings updated.

---------

Signed-off-by: Tomasz Bartczak <kretesenator@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants