Skip to content

Commit

Permalink
Remove templating (#353)
Browse files Browse the repository at this point in the history
Templating of containers has been moved to
https://github.com/DIAGNijmegen/rse-grand-challenge-forge. This repo now
only contains methods related to metrics calculation.

`distance_transform_edt_float32` has been removed now that we are less
memory bound.

Closes #352
  • Loading branch information
jmsmkn authored Aug 7, 2024
1 parent 33573e7 commit b5af1a5
Show file tree
Hide file tree
Showing 75 changed files with 86 additions and 17,324 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
- published

env:
MINIMUM_PYTHON_VERSION: "3.8"
MINIMUM_PYTHON_VERSION: "3.9"

concurrency:
group: ${{ github.head_ref || github.run_id }}
Expand Down Expand Up @@ -36,7 +36,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v3
- name: Setup Python ${{ matrix.python-version }}
Expand Down
14 changes: 7 additions & 7 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.6.0
hooks:
- id: check-docstring-first
- id: debug-statements
- id: end-of-file-fixer
- id: mixed-line-ending
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
rev: v3.17.0
hooks:
- id: pyupgrade
language: python
args: [--py38-plus]
args: [--py39-plus]
- repo: https://github.com/pycqa/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort
- repo: https://github.com/ambv/black
rev: 23.1.0
rev: 24.8.0
hooks:
- id: black
language: python
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
rev: 7.1.1
hooks:
- id: flake8
language: python
Expand All @@ -35,6 +35,6 @@ repos:
- mccabe
- yesqa
- repo: https://github.com/pre-commit/mirrors-mypy
rev: 'v1.0.1'
rev: 'v1.11.1'
hooks:
- id: mypy
6 changes: 6 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# History

## 0.5.0 (2024-08-07)

- Removed templating
- Added support for Python 3.12
- Dropped support for Python 3.8

## 0.4.2 (2023-06-09)

- Replace pandas append function with concat
Expand Down
20 changes: 7 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,20 @@
[![Documentation Status](https://img.shields.io/badge/docs-passing-4a4c4c1.svg)](https://comic.github.io/evalutils/)
[![Code style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)

evalutils helps challenge administrators easily create evaluation
containers for grand-challenge.org.
evalutils contains useful functions for evaluating machine learning models in the context of medical imaging.

- Free software: MIT license
- Documentation: <https://comic.github.io/evalutils/>.

## Features

- Generation your challenge evaluation project boilerplate using
[Cookiecutter](https://github.com/audreyr/cookiecutter)
- Scripts to build, test and export your generated docker container
for grand-challenge.org
- Loading of CSV, ITK and Pillow compatible prediction files
- Validation of submitted predictions
- Interface to SciKit-Learn metrics and Pandas aggregations
- Bounding box annotations with Jaccard Index calculations
- Calculations of bootstrapped ROC curves
- Scoring for detection tasks
- Efficient calculation of confusion matrices, jaccard scores, dice scores, hausdorff distances,
absolute volume differences, and relative volume differences

## Getting Started

[evalutils](https://github.com/comic/evalutils) requires Python 3.8 or
above, and can be installed from `pip`.
Please see the [Getting Started](https://comic.github.io/evalutils/usage.html)
documentation for more details.
[evalutils](https://github.com/comic/evalutils) requires Python 3.9 or above, and can be installed from `pip`.
Please see the [Getting Started](https://comic.github.io/evalutils/usage.html) documentation for more details.
5 changes: 2 additions & 3 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
import os
import sys
from importlib.metadata import version as _get_version
from typing import Dict, List

evalutils_version = _get_version("evalutils")

Expand Down Expand Up @@ -111,7 +110,7 @@ def setup(app):
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path: List[str] = []
html_static_path: list[str] = []


# -- Options for HTMLHelp output ---------------------------------------
Expand All @@ -122,7 +121,7 @@ def setup(app):

# -- Options for LaTeX output ------------------------------------------

latex_elements: Dict[str, str] = {
latex_elements: dict[str, str] = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
Expand Down
114 changes: 0 additions & 114 deletions docs/contributing.rst

This file was deleted.

16 changes: 1 addition & 15 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,28 +1,14 @@
Welcome to evalutils's documentation!
======================================

The automated evaluation system on `Grand Challenge`_ will run an instance of
your evaluation docker container image on each new submission.
The users submission will be extracted on ``/input/``, and your container
is expected to calculate all of the metrics for this submission, and write them
to ``/output/metrics.json``.
If the metrics cannot be calculated, the container should write to ``stderr``
and the last line of this will be passed to the user so that they can debug
their submission.

`evalutils`_ helps you do this by providing a package that helps you create
a project structure, load and validate the submissions, score the submission,
write the json file, and package this in a docker container image so that
you can then upload it to `Grand Challenge`_.
`evalutils`_ contains useful functions for evaluating machine learning models in the context of medical imaging.

.. toctree::
:maxdepth: 2
:caption: Contents:

installation
usage
modules
contributing

Indices and tables
==================
Expand Down
25 changes: 2 additions & 23 deletions docs/modules.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,6 @@
Modules
=======


Evalutils
---------

.. py:module:: evalutils
.. automodule:: evalutils.evalutils
:members:

IO
--

.. automodule:: evalutils.io
:members:

Validators
----------

.. automodule:: evalutils.validators
:members:

Scorers
-------

Expand All @@ -37,8 +16,8 @@ Annotations
.. autoclass:: BoundingBox()
:members:

Statistics
----------
Stats
-----

.. automodule:: evalutils.stats
:members:
Expand Down
Loading

0 comments on commit b5af1a5

Please sign in to comment.