Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Metrics] AUC/AUROC class interface #5479

Merged
merged 42 commits into from
Jan 27, 2021
Merged

[Metrics] AUC/AUROC class interface #5479

merged 42 commits into from
Jan 27, 2021

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Jan 12, 2021 •
edited by Borda
Loading

What does this PR do?

Fixes #4293
Fixes #4547

Implementation of class based interface for AUC and AUROC and unification with their respective functional counterpart.
Brings a number of improvements to AUROC also:

  • auroc and multiclass_auroc are combined into auroc
  • support for multilabel (not sure if we already did this, but now we test for it and state it in the docs)
  • support for multi dim tensors
  • support for different averaging in multiclass setting

This should be one of last PRs before the hole metric package is unified.

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified
  • Check that target branch and milestone match!

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added feature Is an improvement or enhancement Metrics labels Jan 12, 2021
@SkafteNicki SkafteNicki added this to the 1.2 milestone Jan 12, 2021
This was linked to issues Jan 12, 2021
error when using auroc #4293
Closed
Class version of AUROC metric #4547
Closed
marload
marload suggested changes Jan 12, 2021
View reviewed changes
Copy link
Contributor

@marload marload left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great Work 👍
But I have some request changes.

tests/metrics/classification/test_auc.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/utils.py Show resolved Hide resolved
pytorch_lightning/metrics/functional/auroc.py Outdated Show resolved Hide resolved
@SkafteNicki SkafteNicki mentioned this pull request Jan 12, 2021
Merged
@pep8speaks
Copy link

pep8speaks commented Jan 12, 2021 •
edited
Loading

Hello @SkafteNicki! Thanks for updating this PR.

Line 161:65: E231 missing whitespace after ','

Comment last updated at 2021-01-27 10:03:36 UTC

teddykoker
teddykoker approved these changes Jan 12, 2021
View reviewed changes
Copy link
Contributor

@teddykoker teddykoker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant! Thank you so much @SkafteNicki

@Borda Borda added the ready PRs ready to be merged label Jan 24, 2021
tchaton
tchaton approved these changes Jan 24, 2021
View reviewed changes
Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

pytorch_lightning/metrics/classification/auroc.py Show resolved Hide resolved
@mergify mergify bot removed the has conflicts label Jan 26, 2021
@Borda
Copy link
Member

Borda commented Jan 27, 2021

@SkafteNicki mind resolve conflicts? 🐰

@mergify mergify bot removed the has conflicts label Jan 27, 2021
@SkafteNicki SkafteNicki enabled auto-merge (squash) January 27, 2021 08:34
@SkafteNicki SkafteNicki merged commit 221c4a0 into Lightning-AI:release/1.2-dev Jan 27, 2021
This was referenced Jan 27, 2021
Closed
Closed
Closed
@SkafteNicki SkafteNicki deleted the metrics/auc_auroc branch March 2, 2021 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda approved these changes

@teddykoker teddykoker teddykoker approved these changes

@tchaton tchaton tchaton approved these changes

@marload marload marload approved these changes

@ananyahjha93 ananyahjha93 Awaiting requested review from ananyahjha93

@awaelchli awaelchli Awaiting requested review from awaelchli

@edenlightning edenlightning Awaiting requested review from edenlightning

@justusschock justusschock Awaiting requested review from justusschock

@SeanNaren SeanNaren Awaiting requested review from SeanNaren

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

Assignees
No one assigned
Labels
feature Is an improvement or enhancement ready PRs ready to be merged
Projects
None yet
Milestone
pl:1.2
Development

Successfully merging this pull request may close these issues.

Class version of AUROC metric error when using auroc
6 participants
@SkafteNicki @pep8speaks @Borda @teddykoker @tchaton @marload