Refactor/classification 10

[SkafteNicki]

What does this PR do?

Continues work on classification refactor #1001
Prior work:

• [Refactor] Classification 9/n #1189
• [Refactor] Classification 8/n #1175
• [Refactor] Classification 7/n #1167
• [Refactor] Classification 6/n #1163
• [Refactor] Classification 5/n #1159
• [Refactor] Classification 4/n #1151
• [Refactor] Classification 3/n #1145
• [Refactor] Classification 2/n #1143
• [Refactor] Classification 1/n #1054

This PR is introducing the changes for the transition (deprecation) of the old API to the new.

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[Borda]

Shall we also remove all the commented-out code like tests?

[SkafteNicki]

Shall we also remove all the commented-out code like tests?

Yes.
There may be some of the tests that we want to reintroduce but we should still be able to find them in the future.

[Borda]

@SkafteNicki still draft? 😇

[codecov]

Codecov Report

Merging #1197 (84c4f1d) into devel/classification (6b60bf3) will decrease coverage by 39%.
The diff coverage is 17%.

@@                  Coverage Diff                   @@
##           devel/classification   #1197     +/-   ##
======================================================
- Coverage                    89%     50%    -39%     
======================================================
  Files                       191     191             
  Lines                     10786   11201    +415     
======================================================
- Hits                       9577    5632   -3945     
- Misses                     1209    5569   +4360     

[SkafteNicki]

@Borda ready for review

[Borda]

Could you pls check the typing?

[SkafteNicki]

@Borda not sure what to do about the typing. Take jaccard_score as an example:

def jaccard_index(
    preds: Tensor,
    target: Tensor,
    num_classes: int,
    average: Optional[str] = "macro",
    ignore_index: Optional[int] = None,
    absent_score: float = 0.0,
    threshold: float = 0.5,
    task: Optional[Literal["binary", "multiclass", "multilabel"]] = None,  # new arg
    num_labels: Optional[int] = None,  # new arg
    validate_args: bool = True,  # new arg
) -> Tensor:
    if task is not None:
        kwargs = dict(ignore_index=ignore_index, validate_args=validate_args)
        if task == "binary":
            return binary_jaccard_index(preds, target, threshold, **kwargs)
        if task == "multiclass":
            return multiclass_jaccard_index(preds, target, num_classes, average, **kwargs)
        if task == "multilabel":
            return multilabel_jaccard_index(preds, target, num_labels, threshold, average, **kwargs)
        raise ValueError(
            f"Expected argument `task` to either be `'binary'`, `'multiclass'` or `'multilabel'` but got {task}"
        )
    else:
        rank_zero_warn(
            ...
            DeprecationWarning,
        )
    confmat = _confusion_matrix_update(preds, target, num_classes, threshold)
    return _jaccard_from_confmat(confmat, num_classes, average, ignore_index, absent_score)

The new argument num_labels is an optional argument because it is not necessary for the old implementation or in the case of task='binary' and task='multiclass', but it is an required parameter if user set task='multilabel'. Implementation wise should it be pretty safe to call it like this because all arguments are validated inside the sub-functions however it still leads to a type-mismatch.