[REVIEW]: TorchMetrics - Measuring Reproducibility in PyTorch #4101
Comments
whedon
added
Makefile
Python
review
waitlisted
|
Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @inpefess, @richrobe, @reneraab it looks like you're currently assigned to review this paper 🎉. Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post. ⭐ Important ⭐ If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿 To fix this do the following two things:
For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
Wordcount for |
|
PDF failed to compile for issue #4101 with the following error: |
|
|
|
not sure if it is important, but the actual version is |
|
'Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.' --> Lightning-AI/torchmetrics#808 |
yes, the problem seems to be related to PyTorch as they do not provide binaries for python 3.10 yet, and moreover, they suggest using their package warehouse... on the other hand, all shall be well working in the Conda environment >> https://anaconda.org/conda-forge/torchmetrics cc: @justusschock |
|
@taless474 is there a way to have authors without affiliations (@maximsch2 chose not to have an affiliation here)? |
|
@justusschock yes, authors without affiliations are allowed |
|
No, they are not, at least not in the paper processing. See affiliation 3 in https://joss.readthedocs.io/en/latest/submitting.html#example-paper-and-bibliography for how to handle this situation. |
|
Thank you @danielskatz |
|
@whedon generate pdf |
|
FYI, both changes Lightning-AI/torchmetrics#810 and Lightning-AI/torchmetrics#817 were cherry-picked and will be released in the next |
|
The software paper gives a good introduction to TorchMetrics and contains all the required sections. However, overall it may be made slightly more accessible for non-specialist audiences. Since this is my first JOSS review, I can't quite judge how accessible JOSS papers typically are and would leave this judgment to the other reviewers. In terms of language I would suggest the following changes: line 16 in PDF (56 in paper.md): Change "an code and data section" to "a code and data section" line 54 (96): "trained on hundreds of devices GPUs or TPUs". Either remove "GPUs or TPUs" or add "such as" before "GPUs": "trained on hundreds of devices such as GPUs or TPUs". line 60 (98): The first sentence introduces "a functional interface" and the next sentence uses "they" to refer to the functions that this interface provides. This is not immediately obvious language-wise. Consider rephrasing the second sentence to "This interface provides simple Python functions that" |
|
@reneraab thank you for your kind suggestions, all shall be implemented in Lightning-AI/torchmetrics#819 |
|
|
|
👋 @openjournals/joss-eics, this paper is ready to be accepted and published. Check final proof 👉 openjournals/joss-papers#2939 If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2939, then you can now move forward with accepting the submission by compiling again with the flag |
|
@justusschock just one thing, the zenodo archive has a different title and authors than the paper. Typically, these are the same for a JOSS-related paper. Note that you can manually change the metadata of the zenodo archive. |
|
The zendo points to the latest release so it seems that the title will always change according to the release name which is rather dependent on the particular release content... also for having the last edit from Lightning-AI/torchmetrics#830 in a release we would need to make one more @justusschock do you know how to change the zenodo manually?
updated in the repo |
|
@whedon set v0.7.2 as version |
|
I'm sorry @justusschock, I'm afraid I can't do that. That's something only editors are allowed to do. |
|
@taless474 We just released v0.7.2 to address this (and also release all the changes of this review process!) together with the new zenodo doi 10.5281/zenodo.6037875. I'm afraid, I don't have permissions to change this, so I would kindly ask you to do so :) |
|
@whedon set archive as 10.5281/zenodo.6037875 |
|
I'm sorry human, I don't understand that. You can see what commands I support by typing: |
|
@whedon set 10.5281/zenodo.6037875 as archive |
|
OK. 10.5281/zenodo.6037875 is the archive. |
|
@whedon set v0.7.2 as version |
|
OK. v0.7.2 is the version. |
|
Hi @justusschock! A few items to wrap up:
|
|
Hi @kthyng , Thanks a lot! With your points you are completely correct and they have been addressed in Lightning-AI/torchmetrics#836 . Anything else we need to do on this? |
|
@whedon generate pdf |
|
ok looks good! |
|
@whedon accept deposit=true |
|
|
🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦 |
|
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨 Here's what you must now do:
Any issues? Notify your editorial technical team... |
|
Congratulations on your new publication @justusschock! Many thanks to editor @taless474 and reviewers @inpefess, @richrobe, and @reneraab for your time, hard work, and expertise!! |
|
🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉 If you would like to include a link to your paper from your README use the following code snippets: This is how it will look in your documentation: We need your help! Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
|
Submitting author: @justusschock (Justus Schock)
Repository: https://github.com/PyTorchLightning/metrics
Version: v0.7.2
Editor: @taless474
Reviewer: @inpefess, @richrobe, @reneraab
Archive: 10.5281/zenodo.6037875
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@inpefess & @richrobe & @reneraab, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @taless474 know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Review checklist for @inpefess
✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
Review checklist for @richrobe
✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
Review checklist for @reneraab
✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
The text was updated successfully, but these errors were encountered: