JOSS Review

[Het-Shah]

Hi @a-ws-m! This will be an issue for my review for your JOSS submission.

Pros -

• The API is straightforward for the end-user to use.

Cons -

• The library seems to have ~1000 lines of actual code (1165). This begs the question about the substantial scholarly effort.
• The paper lacks details about the library itself. It is unclear as to how the package is structured and what are the key functionalities.
• It would be advantageous if you could make some basic ablations and provide their results. I am unaware of the datasets, but you can try on the smallest dataset and provide results.

Feel free to comment on these. I would be more than happy to discuss the same.

[a-ws-m]

Thanks for the review 👍. I've commented on the first con in the JOSS review thread as it was brought up by multiple reviewers. Regarding the second point, I've added a sentence to clarify some of the packages' features; let me know if there's some particular detail you think should be added, I intended to describe the key functionality within that paragraph. I'll work on adding some benchmarks now; there's a small difficulty in reporting the metrics because the most interesting ones (NLL) have no other values to compare against (to my knowledge) because current benchmarks in materials science don't report uncertainty quantification metrics.

[a-ws-m]

The benchmarks are complete and I've reported the results for a few models with different hyperparameters in the appropriate results directory.

[Het-Shah]

Great! everything seems fine to me now. I have completed the checklist on the JOSS thread and will cross-post this comment there as well.