Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the Perplexity metric #922

Merged
merged 35 commits into from
Jun 30, 2022
Merged

Conversation

bryant1410
Copy link
Contributor

@bryant1410 bryant1410 commented Mar 30, 2022

What does this PR do?

Fixes #<issue_number>

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Mar 30, 2022

Codecov Report

Merging #922 (c0e21aa) into master (b49e579) will decrease coverage by 44%.
The diff coverage is 90%.

@@           Coverage Diff           @@
##           master   #922     +/-   ##
=======================================
- Coverage      94%    50%    -44%     
=======================================
  Files         180    182      +2     
  Lines        8082   8141     +59     
=======================================
- Hits         7596   4059   -3537     
- Misses        486   4082   +3596     

@Borda Borda added this to the v0.8 milestone Mar 31, 2022
docs/source/references/functional.rst Outdated Show resolved Hide resolved
tests/text/test_perplexity.py Outdated Show resolved Hide resolved
torchmetrics/functional/text/perplexity.py Outdated Show resolved Hide resolved
bryant1410 and others added 2 commits March 31, 2022 10:07
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need requires testing 🐰

Copy link
Contributor

@stancld stancld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryant1410 Thank you a lot for contributing! 🎉

I have one issue regarding the API of this new metric so that it will better correspond to the design of torchmetrics.

docs/source/references/modules.rst Outdated Show resolved Hide resolved
torchmetrics/functional/text/perplexity.py Outdated Show resolved Hide resolved
@bryant1410
Copy link
Contributor Author

need requires testing rabbit

@Borda what do you mean? To add more tests? Specifically, testing what behavior?

bryant1410 and others added 2 commits March 31, 2022 14:25
Co-authored-by: Daniel Stancl <46073029+stancld@users.noreply.github.com>
@stancld
Copy link
Contributor

stancld commented Jun 24, 2022

@Borda Looks like InfoLM is now really ready for review, then I hope I'll prepare this one tomorrow O:]

@mergify mergify bot removed the has conflicts label Jun 25, 2022
src/torchmetrics/functional/text/perplexity.py Outdated Show resolved Hide resolved
@mergify mergify bot added the ready label Jun 27, 2022
src/torchmetrics/text/perplexity.py Outdated Show resolved Hide resolved
src/torchmetrics/text/perplexity.py Show resolved Hide resolved
src/torchmetrics/text/perplexity.py Outdated Show resolved Hide resolved
src/torchmetrics/text/perplexity.py Show resolved Hide resolved
tests/unittests/text/test_perplexity.py Outdated Show resolved Hide resolved
tests/unittests/text/test_perplexity.py Outdated Show resolved Hide resolved
@mergify mergify bot removed the ready label Jun 30, 2022
@mergify mergify bot added ready and removed has conflicts labels Jun 30, 2022
@SkafteNicki SkafteNicki merged commit c44aca1 into Lightning-AI:master Jun 30, 2022
@bryant1410 bryant1410 deleted the perplexity branch June 30, 2022 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants