Merge pull request #720 from amcadmus/master

Merge devel into master

.gitignore

@@ -40,3 +40,4 @@ dbconfig.json
 .vscode/*
 .idea/*
 _build
+tests/generator/calypso_test_path

README.md

@@ -92,50 +92,6 @@ To test if the installation is successful, you may execute
 ```bash
 dpgen -h
 ```
-and if everything works, it gives
-```
-DeepModeling
-------------
-Version: 0.9.2
-Date:    Mar-25-2021
-Path:    /root/yuzhi/dpgen-test/lib/python3.6/site-packages/dpgen
-
-Reference
-------------
-Please cite:
-Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
-DP-GEN: A concurrent learning platform for the generation of reliable deep learning
-based potential energy models, Computer Physics Communications, 2020, 107206.
-------------
-
-Description
-------------
-usage: dpgen [-h]
-             {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
-             ...
-
-dpgen is a convenient script that uses DeepGenerator to prepare initial data,
-drive DeepMDkit and analyze results. This script works based on several sub-
-commands with their own options. To see the options for the sub-commands, type
-"dpgen sub-command -h".
-
-positional arguments:
-  {init_surf,init_bulk,auto_gen_param,init_reaction,run,run/report,collect,simplify,autotest,db}
-    init_surf           Generating initial data for surface systems.
-    init_bulk           Generating initial data for bulk systems.
-    auto_gen_param      auto gen param.json
-    init_reaction       Generating initial data for reactive systems.
-    run                 Main process of Deep Potential Generator.
-    run/report          Report the systems and the thermodynamic conditions of
-                        the labeled frames.
-    collect             Collect data.
-    simplify            Simplify data.
-    autotest            Auto-test for Deep Potential.
-    db                  Collecting data from DP-GEN.
-
-optional arguments:
-  -h, --help            show this help message and exit
-```
 
 
 ## Init: Preparing Initial Data
@@ -378,7 +334,7 @@ In each iteration, there are three stages of work, namely, `00.train  01.model_d
 
 + 00.train: DP-GEN will train several (default 4) models based on initial and generated data. The only difference between these models is the random seed for neural network initialization.
 
-+ 01.model_devi : represent for model-deviation. DP-GEN will use models obtained from 00.train to run Molecular Dynamics(default LAMMPS). Larger deviation for structure properties (default is force of atoms) means less accuracy of the models. Using this criterion, a few fructures will be selected and put into next stage `02.fp` for more accurate calculation based on First Principles.
++ 01.model_devi : represent for model-deviation. Model-deviation engine in `01.model_devi` can be chosen between Molecular Dynamics(LAMMPS and GROMACS) or Structures Prediction(CALYPSO). DP-GEN will use models obtained from 00.train to run Molecular Dynamics or to run structure optimization with ASE in CALYPSO. Larger deviation for structure properties (default is force of atoms) means less accuracy of the models. Using this criterion, a few structures will be selected and put into next stage `02.fp` for more accurate calculation based on First Principles.
 
 + 02.fp : Selected structures will be calculated by first principles methods(default VASP). DP-GEN will obtain some new data and put them together with initial data and data generated in previous iterations. After that a new training will be set up and DP-GEN will enter next iteration!
 
@@ -614,7 +570,40 @@ The bold notation of key (such aas **type_map**) means that it's a necessary key
 | *fp_style == cp2k*
 | **user_fp_params** | Dict |  |Parameters for cp2k calculation. find detail in manual.cp2k.org. only the kind section must be set before use.  we assume that you have basic knowledge for cp2k input.
 | **external_input_path** | String |  | Conflict with key:user_fp_params, use the template input provided by user, some rules should be followed, read the following text in detail.
+| *fp_style == ABACUS*
+| **user_fp_params** | Dict |  |Parameters for ABACUS INPUT. find detail [Here](https://github.com/deepmodeling/abacus-develop/blob/develop/docs/input-main.md#out-descriptor). If `deepks_model` is set, the model file should be in the pseudopotential directory. 
+| **fp_orb_files** | List |  |List of atomic orbital files. The files should be in pseudopotential directory. 
+| **fp_dpks_descriptor** | String |  |DeePKS descriptor file name. The file should be in pseudopotential directory. 
+
+One can choose the model-deviation engine by specifying the key `model_devi_engine`. If `model_devi_engine` is not specified, the default model-deviation engine will be LAMMPS.
+
+There are some new keys needed to be added into `param` and `machine` if CALYPSO as model-deviation engine.
 
+The bold notation of key (such as **calypso_path**) means that it's a necessary key.
+
+ Key  | Type          | Example                                                      | Discription                                                      |
+| :---------------- | :--------------------- | :-------------------------------------- | :-------------------------------------------------------------|
+| *in param file*
+| **model_devi_engine** | string | "calypso" | CALYPSO as model-deviation engine.|
+| **calypso_input_path** | string | "/home/zhenyu/workplace/debug" | The absolute path of CALYPSO input file named input.dat(PSTRESS and fmax should be included), when this keys exists, all the iters will use the same CALYPSO input file until reach the number of max iter specified by **model_devi_max_iter** and **model_devi_jobs** key will not work.|
+| **model_devi_max_iter** | int | 10 | The max iter number code can run, it works when **calypso_input_path** exists.|
+| **model_devi_jobs** | List of Dict | [{ "times":[3],"NameOfAtoms":["Al","Cu"],"NumberOfAtoms":[1,10],"NumberOfFormula":[1,2],"Volume":[300],"DistanceOfIon":[[ 1.48,1.44],[ 1.44,1.41]],"PsoRatio":[0.6],"PopSize":[5],"MaxStep":[3],"ICode":[1],"Split":"T"},...] |  Settings for exploration in `01.model_devi`. Different number in `times` List means different iteration index and iterations mentioned in List wil use same CALYPSO parameters.|
+| **model_devi_jobs["times"]** | List of int | [0,1,2] | Different number in `times` List means different iteration index and iterations mentioned in List wil use same CALYPSO parameters.|
+| **model_devi_jobs["NameOfAtoms"]** | List of string |["Al","Cu"] | Parameter of CALYPSO input file, means the element species of structures to be generated. |
+| **model_devi_jobs["NumberOfAtoms"]** | List of int |[1,10] | Parameter of CALYPSO input file, means the number of atoms for each chemical species in one formula unit. |
+| **model_devi_jobs["NumberOfFormula"]** | List of int |[1,2] | Parameter of CALYPSO input file, means the range of formula unit per cell. |
+| **model_devi_jobs["Volume"]** | List of int |[300] | Parameter of CALYPSO input file, means the colume per formula unit(angstrom^3). |
+| **model_devi_jobs["DistanceOfIon"]** | List of float |[[ 1.48,1.44],[ 1.44,1.41]] | Parameter of CALYPSO input file, means minimal distance between atoms of each chemical species. Unit is in angstrom. |
+| **model_devi_jobs["PsoRatio"]** | List of float |[0.6] | Parameter of CALYPSO input file, means the proportion of the structures generated by PSO. |
+| **model_devi_jobs["PopSize"]** | List of int |[5] | Parameter of CALYPSO input file, means the number of structures to be generated in one step in CALYPSO. |
+| **model_devi_jobs["MaxStep"]** | List of int |[3] | Parameter of CALYPSO input file, means the number of max step in CALYPSO.|
+| **model_devi_jobs["ICode"]** | List of int |[13] | Parameter of CALYPSO input file, means the chosen of local optimization, 1 is vasp and 13 is ASE with dp.  |
+| **model_devi_jobs["Split"]** | String |"T" | Parameter of CALYPSO input file, means that generating structures and optimizing structures are split into two parts, in dpgen workflow, Split must be T. |
+| **model_devi_jobs["PSTRESS"]** | List of float |[0.001] | Same as PSTRESS in INCAR. |
+| **model_devi_jobs["fmax"]** | List of float |[0.01] | The convergence criterion is that the force on all individual atoms should be less than *fmax*. |
+| *in machine file*
+| **model_devi["deepmdkit_python"]** | String | "/home/zhenyu/soft/deepmd-kit/bin/python" | A python path with deepmd package. |
+| **model_devi["calypso_path"]** | string | "/home/zhenyu/workplace/debug" | The absolute path of calypso.x.|
 
 #### Rules for cp2k input at dictionary form
    Converting cp2k input is very simple as dictionary used to dpgen input. You just need follow some simple rule:
@@ -1352,7 +1341,7 @@ mem_limit | Interger | 16 | Maximal memory permitted to apply for the job.
 | group_size | Integer | 5 | DP-GEN will put these jobs together in one submitting script.
 | user_forward_files | List of str | ["/path_to/vdw_kernel.bindat"] | These files will be uploaded in each calculation task. You should make sure provide the path exists. 
 | user_backward_files | List of str | ["HILLS"] | Besides DP-GEN's normal output, these files will be downloaded after each calculation. You should make sure these files can be generated.
-  
+
 ## Troubleshooting
 1. The most common problem is whether two settings correspond with each other, including:
     - The order of elements in `type_map` and `mass_map` and **`fp_pp_files`**.

doc/conf.py

@@ -11,6 +11,7 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 import os
+import sys
 import subprocess
 # import sys
 import recommonmark
@@ -79,3 +80,10 @@
 autodoc_default_flags = ['members']
 autosummary_generate = True
 master_doc = 'index'
+
+
+def generate_arginfo(app):
+    subprocess.check_output((sys.executable, "gen_arginfo.py"), universal_newlines=True)
+
+def setup(app):
+    app.connect('builder-inited', generate_arginfo)

doc/gen_arginfo.py

@@ -0,0 +1,5 @@
+from dpgen.generator.arginfo import run_mdata_arginfo
+
+run_mdata_doc = run_mdata_arginfo().gen_doc()
+with open('run-mdata-auto.rst', 'w') as f:
+    f.write(run_mdata_doc)

doc/index.rst

@@ -2,6 +2,14 @@
 DPGEN's documentation
 ==========================
 
+.. _parameters::
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Parameters
+
+   run-mdata.rst
+
 .. _tutorial:
 
 .. toctree::

doc/requirements.txt

@@ -1,6 +1,7 @@
-sphinx==3.2.1
-recommonmark==0.7.1
-sphinx_rtd_theme==0.5.2
-sphinx_markdown_tables==0.0.15
-myst-parser==0.13.7
+sphinx>=4.0.2
+recommonmark
+sphinx_rtd_theme
+sphinx_markdown_tables
+myst-parser
 deepmodeling_sphinx
+.

doc/run-mdata.rst

@@ -0,0 +1,4 @@
+dpgen run machine parameters
+============================
+
+.. include:: run-mdata-auto.rst

dpgen/auto_test/Interstitial.py

@@ -172,6 +172,120 @@ def make_confs(self,
                     dumpfn(self.supercell, 'supercell.json')
                 os.chdir(cwd)
 
+
+                if 'bcc_self' in self.parameter and self.parameter['bcc_self']:
+                    os.chdir(path_to_work)
+                    with open('POSCAR', 'r') as fin:
+                        fin.readline()
+                        scale = float(fin.readline().split()[0])
+                        latt_param = float(fin.readline().split()[0])
+                        latt_param *= scale
+
+                    if not os.path.isfile('task.000000/POSCAR'):
+                        raise RuntimeError("need task.000000 structure as reference")
+
+                    with open('task.000000/POSCAR', 'r') as fin:
+                        pos_line = fin.read().split('\n')
+
+                    super_latt_param = float(pos_line[2].split()[0])
+
+                    output_task1 = os.path.join(path_to_work, 'task.%06d' % (len(dss)))
+                    os.makedirs(output_task1, exist_ok=True)
+                    os.chdir(output_task1)
+                    task_list.append(output_task1)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float(latt_param/4/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' 0.000000 ' + self.insert_ele[0]
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc tetrahedral')
+                    os.chdir(cwd)
+
+                    output_task2 = os.path.join(path_to_work, 'task.%06d' % (len(dss)+1))
+                    os.makedirs(output_task2, exist_ok=True)
+                    os.chdir(output_task2)
+                    task_list.append(output_task2)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float(latt_param/2/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' 0.000000 ' + self.insert_ele[0]
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc octahedral')
+                    os.chdir(cwd)
+
+                    output_task3 = os.path.join(path_to_work, 'task.%06d' % (len(dss)+2))
+                    os.makedirs(output_task3, exist_ok=True)
+                    os.chdir(output_task3)
+                    task_list.append(output_task3)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float(latt_param/4/super_latt_param) + ' ' + '%.6f' % float(latt_param/4/super_latt_param) + ' ' + '%.6f' % float(latt_param/4/super_latt_param) + ' ' + self.insert_ele[0]
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc crowdion')
+                    os.chdir(cwd)
+
+                    for idx, ii in enumerate(pos_line):
+                        ss = ii.split()
+                        if len(ss) > 3:
+                            if abs(latt_param/2/super_latt_param - float(ss[0])) < 1e-5 and abs(latt_param/2/super_latt_param - float(ss[1])) < 1e-5 and abs(latt_param/2/super_latt_param - float(ss[2])) < 1e-5:
+                                replace_label = idx
+
+                    output_task4 = os.path.join(path_to_work, 'task.%06d' % (len(dss)+3))
+                    os.makedirs(output_task4, exist_ok=True)
+                    os.chdir(output_task4)
+                    task_list.append(output_task4)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float(latt_param/3/super_latt_param) + ' ' + '%.6f' % float(latt_param/3/super_latt_param) + ' ' + '%.6f' % float(latt_param/3/super_latt_param) + ' ' + self.insert_ele[0]
+                    pos_line[replace_label] = '%.6f' % float(latt_param/3*2/super_latt_param) + ' ' + '%.6f' % float(latt_param/3*2/super_latt_param) + ' ' + '%.6f' % float(latt_param/3*2/super_latt_param) + ' ' + self.insert_ele[0]
+
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc <111> dumbbell')
+                    os.chdir(cwd)
+
+                    output_task5 = os.path.join(path_to_work, 'task.%06d' % (len(dss)+4))
+                    os.makedirs(output_task5, exist_ok=True)
+                    os.chdir(output_task5)
+                    task_list.append(output_task5)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float((latt_param+2.1/2**0.5)/2/super_latt_param) + ' ' + '%.6f' % float((latt_param-2.1/2**0.5)/2/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' ' + self.insert_ele[0]
+                    pos_line[replace_label] = '%.6f' % float((latt_param-2.1/2**0.5)/2/super_latt_param) + ' ' + '%.6f' % float((latt_param+2.1/2**0.5)/2/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' ' + self.insert_ele[0]
+
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc <110> dumbbell')
+                    os.chdir(cwd)
+
+                    output_task6 = os.path.join(path_to_work, 'task.%06d' % (len(dss)+5))
+                    os.makedirs(output_task6, exist_ok=True)
+                    os.chdir(output_task6)
+                    task_list.append(output_task6)
+                    with open(insert_element_task, 'a+') as fout:
+                        print(self.insert_ele[0], file=fout)
+                    dumpfn(self.supercell, 'supercell.json')
+                    pos_line[-2] = '%.6f' % float(latt_param/2/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' ' + '%.6f' % float((latt_param-2.1)/2/super_latt_param) + ' ' + self.insert_ele[0]
+                    pos_line[replace_label] = '%.6f' % float(latt_param/2/super_latt_param) + ' ' + '%.6f' % float(latt_param/2/super_latt_param) + ' ' + '%.6f' % float((latt_param+2.1)/2/super_latt_param) + ' ' + self.insert_ele[0]
+
+                    with open('POSCAR', 'w+') as fout:
+                        for ii in pos_line:
+                                print(ii, file=fout)
+                    print('gen bcc <100> dumbbell')
+                    os.chdir(cwd)
+
+
         return task_list
 
     def post_process(self, task_list):

dpgen/auto_test/Lammps.py

@@ -289,7 +289,7 @@ def compute(self,
                             if len(line) and str(ii) == line[0]:
                                 try:
                                     [float(kk) for kk in line]
-                                except:
+                                except Exception:
                                     continue
                                 stress.append([])
                                 virial.append([])