Better user interface and consistency check through out env/gp/kernel modules #184

nw13slx · 2020-06-11T22:07:47Z

Major changes:

Input arguments:

cutoff is no longer an array. but a dictionary {'twobody': 7, 'threebody':4.5}
kernel_name, kernel functions are all removed. replace by "kernels" that takes an array of all different kernels ['twobody', 'threebody', 'manybody']

new class:

Parameters class and methods that check whether hyper-parameters, hyper-parameter-mask, cutoffs, kernels are consistent. All relevant checks and info retrival in GP, kernel/utils, mgp are shifted to the parameters.py . (hopefully it's easier to maintain.)

special kernel for MGP that loops over grid point, xyz, and triplets inside the kernel.
all prints are replaced by logger. Output class is rewritten with logger. So it can be easier to print to screen and files at the same time. As a result, the verbose level of GPFA and OTF are all changed from "1-5" to "error, warning, info, debug"

Minor changes:

backward_arguments and backward_attributes are added to GP. All old changes in argument and attributes over different versions of Flare are documented there. So old json and pickle file can still be loaded.
a couple more functions are added to GP class to reduce repeated lines
OTF and GPFA header printing, parser, GP reconstruction are slightly revised to make it easier to maintain.
unit tests were adapted

An example of the new GP initialization:

from flare.gp import GaussianProcess
from flare.utils.parameter_helper import ParameterHelper

pm = ParameterHelper(species=['Cu', 'C', 'H', 'O'],
                          kernels={'twobody':[['*', '*'], ['Cu','Cu']],
                                  'threebody':[['*', '*', '*'], ['Cu','Cu', 'Cu']]},
                           parameters={'twobody0':[1.29, 1.47], 'twobody1':[7e-3, 0.19],
                                 'threebody0':[8e-4, 0.89], 'threebody1':[6.1e-3, 0.72],
                                 'cutoff_twobody':7.0, 'cutoff_threebody':4.5,
                                 'noise':0.05, 'energy_noise':0.01})
hm = pm.as_dict()
cutoffs = hm['cutoffs']
hyps = hm['hyps']
hyp_labels = hm['hyp_labels']
kernels = hm['kernels']
 
gp_model = GaussianProcess(kernels=kernels,
                                                 hyps=hyps, cutoffs=cutoffs,
                                                 hyp_labels=hyp_labels,
                                                 hyps_mask=hm)

… into user_interface_refactory Conflicts: flare/mgp/mgp.py

…flare into user_interface_refactory Conflicts: flare/env.py flare/mgp/mgp.py tests/test_env.py

… into user_interface_refactory

…ce spc_set with 'sort'

jonpvandermause · 2020-06-16T20:10:32Z

flare/otf.py

+                if std_in_bound:
+                    new_pos = self.md_step()
+
+                else:
                    # record GP forces
                    self.update_temperature(new_pos)


The logic is wrong here; velocities get sent to zero, since new_pos hasn't been updated. (I'll fix this)

8544b9f addresses this by mirroring the otf loop on master (new positions are calculated first, and then is_std_in_bound is called)

jonpvandermause · 2020-06-16T20:48:54Z

flare/ase/otf.py

+        FLARE_Calculator or DFT calculator
+        '''
+        self.md.step()
+        return self.atoms.positions


This is different from the md_step method in otf.py, which computes the new positions but doesn't overwrite the positions attribute of the structure

I've attempted to resolve this with 125c08f by removing the second call to md.step() in the otf loop

jonpvandermause · 2020-06-16T22:11:11Z

flare/ase/otf.py

+        Get new position in molecular dynamics based on the forces predicted by
+        FLARE_Calculator or DFT calculator
+        '''
+        self.md.step()


Somehow, self.md.step() updates the positions of self.structure, even though it should act only on self.atoms. This causes the velocities and temperatures to be reported as zero in the OTF output file.

1829b0c seems to give a temporary fix to this problem by manually resetting the positions of self.structure. But I'm still confused why self.md.step() updates self.structure.positions...

The issue was that self.md.step() returned atoms.positions rather than a copy of it, which caused self.atoms and self.structure to get updated together downstream. a2a785a fixes this

…up/flare into user_interface_refactory

… into user_interface_refactory

stevetorr · 2020-06-18T15:15:42Z

flare/gp.py

+        kernels (list, optional): Determine the type of kernels. Example:
+            ['2', '3'], ['2', '3', 'mb'], ['2']. Defaults to ['2', '3']


This was a nice change.

stevetorr · 2020-06-18T15:26:51Z

flare/gp.py

@@ -682,33 +625,29 @@ def as_dict(self):

        return out_dict

+    def sync_data(self):


Maybe we should move the setting of the NP training labels to this function, so that it's only a matter of adding to training data and training labels?

Updating the 'all labels' attribute could also happen here.

stevetorr · 2020-06-18T15:28:26Z

flare/gp.py

-        kernel, grad, ek, efk = str_to_kernel_set(kernel_name, multihyps)
+    def update_kernel(self, kernels, component="mc", hyps_mask=None):
+        kernel, grad, ek, efk = str_to_kernel_set(
+            kernels, component, hyps_mask)


No docstring & typehints? :)

stevetorr · 2020-06-18T15:35:06Z

flare/gp.py

@@ -962,3 +933,76 @@ def __del__(self):
        if (self.name in _global_training_labels):
            _global_training_labels.pop(self.name, None)
            _global_training_data.pop(self.name, None)
+
+    @staticmethod
+    def backward_arguments(kwargs, new_args={}):


Love these deprecation warnings. It's great we have this! Will save a lot of grief in using old scripts

stevetorr · 2020-06-18T15:35:59Z

flare/gp_from_aimd.py

@@ -74,7 +75,7 @@ def __init__(self, frames: List[Structure],
                 max_trains: int = np.inf,
                 min_atoms_per_train: int = 1,
                 shuffle_frames: bool = False,
-                 verbose: int = 1,
+                 verbose: str = "INFO",


Great call to re-do it this way

flare/kernels/kernels.py

stevetorr · 2020-06-18T15:43:02Z

flare/mgp/utils.py

+#            triplet1 = array([r1, r2, r12])
+#            triplet2 = array([r2, r1, r12])
+#
+#            if spc1 <= spc2:
+#                spcs = [ctype, spc1, spc2]
+#            else:
+#                spcs = [ctype, spc2, spc1]
+#
+#            triplet = [triplet1, triplet2]
+#            coord = [c1, c2] 
+#
+#            if spcs not in exist_species:
+#                exist_species.append(spcs)
+#                tris += [triplet]
+#                tri_dir += [coord]
+#            else:
+#                k = exist_species.index(spcs)
+#                tris[k] += triplet
+#                tri_dir[k] += coord


Any reason why these guys are still here?

stevetorr · 2020-06-18T15:54:55Z

tests/test_otf.py

+print('running test_otf.py')
+print('current working directory:')
+print(os.getcwd())


I suspect these were accidentally left over from debugging?

stevetorr · 2020-06-18T15:55:18Z

tests/test_otf.py

+    print('running test_otf.py')
+    print('current working directory:')
+    print(os.getcwd())


Likewise :)

stevetorr · 2020-06-18T16:03:01Z

LGTM (looks good to me). Major kudos to @nw13slx and @YuuuuXie for all their blood, sweat, and tears in making FLARE easier to use and maintain. This is really important work that often goes underappreciated.

nw13slx and others added 30 commits May 19, 2020 20:59

change to logger, change init arguments

4763bdd

revise the whole hyps structure

81680a9

update cutoff to dictionary for AtomicEnvironment

d5dd2f7

simplify from mask to args

babe7f2

fix bugs for kernels

a6a7e88

ase logger changed to inherit from Output module

ed5277a

fix mc_sephyps_test and autopep8

640bfdb

upgrade unittest and fix backward compatability

2fcfa32

fix module import err

133cabb

add option 'rank' for predict

83e623a

fix numba type error

7abe7d5

change to verbose name

0094c21

switch back to hyps_mask

ecb02a4

Merge branch 'user_interface_refactory' of github.com:mir-group/flare…

97c87af

… into user_interface_refactory Conflicts: flare/mgp/mgp.py

try logging

24a320d

try to patch the typint glitch

76ab4b1

Merge branch 'feature/lixin/multiple-cutoff' of github.com:mir-group/…

2743868

…flare into user_interface_refactory Conflicts: flare/env.py flare/mgp/mgp.py tests/test_env.py

fix merge error with the autosweep

8b81729

swap the order of converting array to dict

87a003b

fix flush

b551995

fix multihyps and supercount merge error

ff16224

remove mgp_en

b18d453

fix bug in hyps_mask update

a61f8a8

fix numba error & separate get_*_array into utils

c84b86b

update arguments of str_to_kernel_set

ae9ba34

fix backward compatability

7eb7df8

update tests

6b9bfbd

update otf and parser

a4f3208

Merge branch 'user_interface_refactory' of github.com:mir-group/flare…

0843dba

… into user_interface_refactory

fix missing return

33fc68b

nw13slx and others added 2 commits June 16, 2020 10:11

Merge branch 'user_interface_refactory' of github.com:mir-group/flare…

941afad

… into user_interface_refactory

add cell to struc (from_ase_atoms) & fix load_grid bug in mgp & repla…

43dc506

…ce spc_set with 'sort'

jonpvandermause reviewed Jun 16, 2020

View reviewed changes

fix otf logic

8544b9f

jonpvandermause reviewed Jun 16, 2020

View reviewed changes

remove second md step from otf loop

125c08f

jonpvandermause reviewed Jun 16, 2020

View reviewed changes

jonpvandermause and others added 8 commits June 16, 2020 18:13

quick fix for changing structure positions

1829b0c

have md.step return a copy of atom positions

a2a785a

fix one mistake in docs

a53d3db

change default value

1ed4f69

Merge branch 'user_interface_refactory' of https://github.com/mir-gro…

4511a32

…up/flare into user_interface_refactory

sort test_gp_algebra

3af518b

Merge branch 'user_interface_refactory' of github.com:mir-group/flare…

3b1d347

… into user_interface_refactory

fix bug in test_force that skipped all tests

ed5bc7e

stevetorr reviewed Jun 18, 2020

View reviewed changes

flare/kernels/kernels.py Outdated Show resolved Hide resolved

Fix minor typo in docstring

0d087fc

stevetorr reviewed Jun 18, 2020

View reviewed changes

rm comment lines

8411d30

stevetorr reviewed Jun 18, 2020

View reviewed changes

rm comment lines

62f98ea

stevetorr reviewed Jun 18, 2020

View reviewed changes

YuuuXie merged commit 8395cd6 into development Jun 18, 2020

YuuuXie deleted the user_interface_refactory branch June 19, 2020 17:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better user interface and consistency check through out env/gp/kernel modules #184

Better user interface and consistency check through out env/gp/kernel modules #184

nw13slx commented Jun 11, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

jonpvandermause Jun 16, 2020

stevetorr Jun 18, 2020 •

edited

Loading

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020 •

edited

Loading

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020

stevetorr Jun 18, 2020

stevetorr commented Jun 18, 2020 •

edited

Loading

		kernels (list, optional): Determine the type of kernels. Example:
		['2', '3'], ['2', '3', 'mb'], ['2']. Defaults to ['2', '3']

		@@ -682,33 +625,29 @@ def as_dict(self):

		return out_dict

		def sync_data(self):

Better user interface and consistency check through out env/gp/kernel modules #184

Better user interface and consistency check through out env/gp/kernel modules #184

Conversation

nw13slx commented Jun 11, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevetorr Jun 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevetorr Jun 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevetorr commented Jun 18, 2020 • edited Loading

stevetorr Jun 18, 2020 •

edited

Loading

stevetorr Jun 18, 2020 •

edited

Loading

stevetorr commented Jun 18, 2020 •

edited

Loading