Deep Quality Estimation

Quality prediction for brain tumor segmentation on a scale ranging from ⭐ 1 star to ⭐⭐⭐⭐⭐⭐ 6 stars inspired by the paper Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings.
This can be used to estimate the quality of a BraTS glioma segmentation for evaluation purposes or, e.g., as part of a loss function during model training.

Important notes

Important

This package expects images in atlas space and segmentation labels in brats style, i.e.

• label 1 is the necrotic and non-enhancing tumor core
• label 2 is the peritumoral edema
• label 3 is the GD-enhancing tumor (used to be label 4 in older data; both are supported)

Note

The model in this package differs from the one presented in the paper.
Unlike the original model it is trained based on individual radiologists' ratings enabling it to learn the variance between radiologists' estimates.
It outperforms the model presented in the paper on the test set.

Caution

The model is biased to overestimate segmentation quality as it was mainly trained on high-quality segmentations and was exposed to only a few bad samples. We still argue that high scores can be useful.

Installation

With a Python 3.9+ environment, you can install deep_quality_estimation directly from PyPI:

pip install deep_quality_estimation

Use Cases and Tutorials

A minimal example to predict the quality of a segmentation could look like this:

from deep_quality_estimation import DQE

# shown parameters are default values but can be adapted to usecase
dqe = DQE(device="cuda", cuda_devices="0") 

# inputs can be Paths (str or pathlib.Path object), NumPy NDArrays or a mix
mean_score, scores_per_view = dqe.predict(
    t1c="t1c.nii.gz",
    t1="t1.nii.gz",
    t2="t2.nii.gz",
    flair="flair.nii.gz",
    segmentation="segmentation.nii.gz",
)

Citation

If you use deep_quality_estimation in your research, please cite it to support the development!

https://arxiv.org/abs/2205.10355

@misc{kofler2022deepqualityestimationcreating,
      title={Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings}, 
      author={Florian Kofler and Ivan Ezhov and Lucas Fidon and Izabela Horvath and Ezequiel de la Rosa and John LaMaster and Hongwei Li and Tom Finck and Suprosanna Shit and Johannes Paetzold and Spyridon Bakas and Marie Piraud and Jan Kirschke and Tom Vercauteren and Claus Zimmer and Benedikt Wiestler and Bjoern Menze},
      year={2022},
      eprint={2205.10355},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2205.10355}, 
}

Contributing

We welcome all kinds of contributions from the community!

Reporting Bugs, Feature Requests and Questions

Please open a new issue here.

Code contributions

Nice to have you on board! Please have a look at our CONTRIBUTING.md file.