Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JOSS review #38

Merged
merged 112 commits into from
Jul 1, 2022
Merged
Changes from 1 commit
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
159385f
Update citation key
a-ws-m Oct 14, 2021
31b43e9
Add maths to NLL documentation (#33)
a-ws-m Oct 14, 2021
1de84ff
Add coveragerc (#28)
a-ws-m Oct 14, 2021
825d0c7
Correct link to issue comment (#21)
a-ws-m Oct 14, 2021
62a2c57
#24
a-ws-m Oct 14, 2021
335c2c4
Add instructions for utilities (#29)
a-ws-m Oct 14, 2021
7c45497
Make save_struct_data public (#27)
a-ws-m Oct 14, 2021
80644f6
Actually test downloading (#22)
a-ws-m Oct 14, 2021
0bcbebb
Exclude large files from workspace watch
a-ws-m Oct 14, 2021
91a3651
Add pytest-cov to tox deps
a-ws-m Oct 14, 2021
4fd766d
Correct package name
a-ws-m Oct 14, 2021
20a5957
Use pytest.approx for floating points (#23)
a-ws-m Oct 14, 2021
c2cfbf1
Remove reference to tensorflow issue
a-ws-m Oct 14, 2021
1112af1
Correct pytest.approx syntax (#23)
a-ws-m Oct 14, 2021
37ea3bf
Test reload functionality in load_data
a-ws-m Oct 14, 2021
fafe25a
[WIP] Metrics testing suite
a-ws-m Oct 15, 2021
e7b4cb2
Citation (#24)
a-ws-m Oct 19, 2021
2b26f1b
Change RTD Slug (#20)
a-ws-m Oct 21, 2021
5683f30
Add save_struct_data to docs (#27)
a-ws-m Oct 21, 2021
af8f028
Add pytest-mock to test installs
a-ws-m Oct 21, 2021
7c5c17d
Hide metrics tests whilst WIP
a-ws-m Oct 21, 2021
fb54187
Split up model tests (#25)
a-ws-m Oct 21, 2021
a2ac525
Correct variation description
a-ws-m Oct 21, 2021
de19247
Add metrics tests (#26)
a-ws-m Oct 21, 2021
e22576e
Don't cover backwards compatible checks (use tox)
a-ws-m Oct 22, 2021
46b0911
Add a test for custom kernels
a-ws-m Oct 22, 2021
4298d05
Clarify amplitude and length_scale (#32)
a-ws-m Oct 22, 2021
57e793c
Correct TestKernel implementation to match docs
a-ws-m Oct 22, 2021
7072967
Add amplitude and ls to docs (#32)
a-ws-m Oct 22, 2021
ba6c627
[FIX] amplitude and ls property types (#32)
a-ws-m Oct 22, 2021
4a746c1
[WIP] Warnings suppresion
a-ws-m Oct 23, 2021
1dad84a
Warnings handling (#30)
a-ws-m Oct 25, 2021
111cda1
Target audience (openjournals/joss-reviews#3793)
a-ws-m Oct 25, 2021
65ceb43
Add freezing/thawing tests (#26)
a-ws-m Oct 25, 2021
e1cea74
Add black to workspace formatter settings
a-ws-m Oct 25, 2021
99579d4
MEGNet utils testing (#26)
a-ws-m Oct 25, 2021
5b68426
Add clarifying info on package structure (#19)
a-ws-m Oct 25, 2021
d86bab5
[WIP] Benchmarking for MP2019
a-ws-m Oct 26, 2021
75edfc7
Enable training and inference with graphs
a-ws-m Oct 26, 2021
540dfca
Merge pull request #34 from a-ws-m:feature/graph_training
a-ws-m Oct 26, 2021
98c27f3
Update kwargs to reflect possible graph input
a-ws-m Oct 26, 2021
cfe1c38
Update benchmark script to use graphs
a-ws-m Oct 26, 2021
c7be46d
Don't use band gap
a-ws-m Oct 26, 2021
5e3aff8
Fix case where no inducing points are set
a-ws-m Oct 26, 2021
f52ee58
Fix datetime call
a-ws-m Oct 27, 2021
a4e99b0
Add df debug statement
a-ws-m Oct 27, 2021
9cafceb
Training data debug
a-ws-m Oct 27, 2021
0354d8c
Validation data debug
a-ws-m Oct 27, 2021
cf64388
Correct dataframe queries
a-ws-m Oct 27, 2021
2ec48e7
Fix docstring
a-ws-m Oct 27, 2021
caaf935
More debug info
a-ws-m Oct 27, 2021
c4898f0
Fix for indexing error
a-ws-m Oct 27, 2021
d6b1a00
Add training component switch
a-ws-m Nov 3, 2021
db1ca68
Debugging weight names
a-ws-m Nov 3, 2021
861e880
Fix Model reference
a-ws-m Nov 3, 2021
82f0965
Enable loading using string path
a-ws-m Nov 17, 2021
51a2cbe
More debugging of weight names
a-ws-m Nov 17, 2021
2cd344a
Debugging: don't freeze layers
a-ws-m Nov 17, 2021
cfb786e
More fine-grained debugging
a-ws-m Nov 17, 2021
0d2ba0a
Remove obsolete recompilation for inference
a-ws-m Nov 17, 2021
e55e592
More debugging statements
a-ws-m Nov 17, 2021
3d8722a
Debugging statements
a-ws-m Nov 17, 2021
bade67d
Hardcode freezing logic
a-ws-m Nov 17, 2021
546238d
Debug flag
a-ws-m Nov 17, 2021
dba8c18
Fallback KL weight to current one on recompile
a-ws-m Nov 18, 2021
b1fb445
Fix reference to old kwarg
a-ws-m Nov 19, 2021
3dbec5d
Revert to flexible freezing + add checks
a-ws-m Nov 19, 2021
0435d66
Refactoring + change default training batch size
a-ws-m Nov 22, 2021
5849b66
Add weights duplication test
a-ws-m Nov 22, 2021
c5add8f
Boilerplate model benchmarking code
a-ws-m Nov 22, 2021
1109152
Rename benchmarking utils
a-ws-m Nov 22, 2021
1ad70f9
Matbenching boilerplate
a-ws-m Nov 22, 2021
cf9eb66
Matbench perovskites and phonons benchmarks
a-ws-m Nov 22, 2021
0ffac10
Fix benchmark scripts
a-ws-m Nov 22, 2021
a0c5d39
Validation metrics and tuning
a-ws-m Nov 22, 2021
3b6a3eb
Add batch size arguments
a-ws-m Nov 22, 2021
0e6202e
Update default megnet metric
a-ws-m Nov 23, 2021
921d7ee
Fix series types
a-ws-m Nov 23, 2021
9706f15
Replace checkpoint loading attr with CLI arg
a-ws-m Dec 2, 2021
f25ab6a
Make root_dir a command line argument
a-ws-m Dec 13, 2021
c536ca4
Add data_only flag
a-ws-m Dec 13, 2021
b21e2f7
Fix data-only destination
a-ws-m Dec 13, 2021
12e248b
Add kanban to recommended extensions
a-ws-m Dec 13, 2021
a80412b
Replace kanban extension
a-ws-m Dec 13, 2021
e9efe33
Add kanbn
a-ws-m Dec 13, 2021
0c6415a
Fix required fold for data only mode
a-ws-m Dec 18, 2021
e47159b
Fix typo
a-ws-m Dec 18, 2021
ea39205
Fix folding issue
a-ws-m Dec 18, 2021
60b255a
Fix ArgumentError arguments
a-ws-m Jan 18, 2022
f08e79e
Merge branch 'joss-review' of github.com:a-ws-m/unlockNN into joss-re…
a-ws-m Jan 18, 2022
74ce5d8
Fix model dirname method
a-ws-m Jan 19, 2022
59ef78b
Fix ignore_ckpt CLI arg
a-ws-m Feb 11, 2022
c92881d
Fix numpy array inputs to `predict`
a-ws-m Feb 11, 2022
f69d96f
Fix validation metrics path
a-ws-m Feb 11, 2022
c3a0da4
Perovskites benchmark results + vis
a-ws-m Feb 12, 2022
d8faf9b
Update Kanban
a-ws-m Mar 2, 2022
4fe0a86
Results visualisation
a-ws-m Mar 2, 2022
221d7ca
Fix MEGNet metric evaluation
a-ws-m Mar 17, 2022
064a080
Add phonons results + metrics
a-ws-m Apr 5, 2022
ad52a2e
Write probabilistic predictions to disk for `eval`
a-ws-m Jun 23, 2022
40acdc3
Fix targets to write
a-ws-m Jun 23, 2022
2afaf50
[WIP] Uncertainty results visualisation
a-ws-m Jun 23, 2022
12fd4fa
[WIP] First draft of UQ plot code
a-ws-m Jun 24, 2022
0d9389b
Move graphs to appropriate directory
a-ws-m May 3, 2022
61b90dd
Remove obsolete kanban
a-ws-m May 3, 2022
ff68ea6
Compile when loading
a-ws-m Jun 28, 2022
368f18e
Add model predictions
a-ws-m Jun 28, 2022
7b640c3
Calibration and parity plots
a-ws-m Jun 28, 2022
b78256b
Get average metrics and show legend
a-ws-m Jun 30, 2022
a8e5fc9
[WIP] Add benchmarks info
a-ws-m Jun 30, 2022
45ec0e2
Finalise benchmarks info JOSS Review #19
a-ws-m Jun 30, 2022
e2090c8
Clarify parity plots
a-ws-m Jun 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix model dirname method
a-ws-m committed Jan 19, 2022
commit 74ce5d8894ac00b7bf8425b2869ca737e45f9fcd
13 changes: 6 additions & 7 deletions benchmarking/utils.py
Original file line number Diff line number Diff line change
@@ -370,11 +370,10 @@ def load_data(self, download_only: bool = False) -> Optional[Dataset]:
"""Load data from disk for training/evaluation."""
...

@property
def model_dirname(self) -> str:
def model_dirname(self, include_points: bool = True) -> str:
"""Get the name of the model's directory."""
dirname = self.task_name
if not self.meg:
if include_points:
dirname += f"-{self.points}"
if self.fold is not None:
dirname += f"-{self.fold}"
@@ -383,28 +382,28 @@ def model_dirname(self) -> str:
@property
def meg_model_dir(self) -> Path:
"""Get the directory of the MEGNetModel specified by command line args."""
return self.megnet_models_dir / self.model_dirname
return self.megnet_models_dir / self.model_dirname(False)

@property
def model_dir(self) -> Path:
"""Get the models directory for the model specified by command line args."""
return (
self.megnet_models_dir if self.meg else self.prob_models_dir
) / self.model_dirname
) / self.model_dirname(not self.meg)

@property
def log_dir(self) -> Path:
"""Get the logs directory for the model model specified by command line args."""
return (
self.megnet_logs_dir if self.meg else self.prob_logs_dir
) / self.model_dirname
) / self.model_dirname(not self.meg)

@property
def checkpoint_dir(self) -> Path:
"""Get the checkpoint file path."""
return (
self.megnet_models_dir if self.meg else self.prob_models_dir
) / f"{self.model_dirname}-ckpt.h5"
) / f"{self.model_dirname(not self.meg)}-ckpt.h5"

@property
def test_result_path(self) -> Path: