Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smart update of metric collection #709

Merged
merged 51 commits into from
Feb 8, 2022
Merged

Smart update of metric collection #709

merged 51 commits into from
Feb 8, 2022

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Jan 4, 2022

What does this PR do?

Fixes #143

Update MetricCollection such that metrics that have similar metric states (Precision and Recall for example) we only calculate the metric state for one of them and then broadcast it to all other metrics with similar metric states.
After the first call to update we checks if the metric states matches between any metrics. If they do we merge them into a compute group of metrics where we will only update the first and broadcast the state to the remaining metrics in the group

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added the enhancement New feature or request label Jan 4, 2022
@mergify mergify bot added the has conflicts label Jan 4, 2022
@SkafteNicki SkafteNicki marked this pull request as draft January 4, 2022 19:52
@Borda Borda added Important milestonish and removed has conflicts labels Jan 4, 2022
@Borda Borda added this to the v0.8 milestone Jan 4, 2022
@Borda Borda force-pushed the master branch 3 times, most recently from 3a0f7dc to 317182c Compare January 10, 2022 13:05
@Borda Borda mentioned this pull request Jan 20, 2022
4 tasks
torchmetrics/__init__.py Outdated Show resolved Hide resolved
torchmetrics/__init__.py Outdated Show resolved Hide resolved
torchmetrics/__init__.py Outdated Show resolved Hide resolved
@mergify mergify bot removed the has conflicts label Feb 8, 2022
Copy link
Member

@justusschock justusschock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :) added some more text on potential slowdowns

docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
Borda and others added 3 commits February 8, 2022 10:23
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
CHANGELOG.md Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Show resolved Hide resolved
tests/bases/test_collections.py Show resolved Hide resolved
torchmetrics/collections.py Outdated Show resolved Hide resolved
torchmetrics/collections.py Outdated Show resolved Hide resolved
@justusschock
Copy link
Member

Seems sphinx doesn't know :param:, should we try :attr: instead? Param may only work when being within this function...

@SkafteNicki
Copy link
Member Author

@justusschock I changed the text to not include a hyperlink but instead just reference the class docs that are just below the sentence

@mergify mergify bot added the ready label Feb 8, 2022
CHANGELOG.md Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
docs/source/pages/overview.rst Outdated Show resolved Hide resolved
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
@Borda Borda merged commit 7e7cdad into master Feb 8, 2022
@Borda Borda deleted the smart_collection branch February 8, 2022 20:22
@tchaton
Copy link
Contributor

tchaton commented Feb 9, 2022

@SkafteNicki Beautiful addition !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Important milestonish ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Smart update of Collection of CompositionalMetrics
4 participants