add the explanation of autotest to doc

doc/Autotest/Auto-test-to-do-list.md

@@ -0,0 +1,35 @@
+For the implementation, one should do :
+1. Clearly know the input/output of the function/class. How to handle exceptions.
+2. Finish coding
+3. Provide Unittest
+4. Provide Document: what does the user provide in each section of the parameter file (json format)
+
+common.py
+- make_*
+- run_*
+- post_*
+
+Property
+- EOS
+- Elastic
+- Vacancy
+- Interstitial
+- Surface
+
+
+
+Task:
+- VASP
+- DEEPMD_LMP
+- MEAM_LMP
+
+
+Specific functions:
+1. Property.make_confs : Make configurations needed to compute the property. 
+        The tasks directory will be named as path_to_work/task.xxxxxx
+        IMPORTANT: handel the case when the directory exists.
+2. Property.cmpt :  Compute the property. 
+3. Task.make_input_file(Property.task_type): Prepare input files for a computational task.
+        For example, the VASP prepares INCAR.
+        LAMMPS (including DeePMD, MEAM...) prepares in.lammps.
+        The parameter of this task will be stored in 'output_dir/task.json'

doc/Autotest/Auto-test.md

@@ -0,0 +1,122 @@
+Suppose that we have a potential (can be DFT, DP, MEAM ...), `autotest` helps us automatically calculate M porperties on N configurations. The folder where the `autotest` runs is called the `autotest`'s working directory. Different potentials should be tested in different working directories.
+
+A property is tested in three stages: `make`, `run` and `post`. `make` prepare all computational tasks that are needed to calculate the property. For example to calculate EOS, `autotest` prepare a series of tasks, each of which has a scaled configuration with certain volume, and all necessary input files necessary for starting a VAPS or LAMMPS relaxation. `run` sends all the computational tasks to remote computational resources defined in a machine configuration file like `machine.json`, and automatically collect the results when remote calculations finish. `post` calculates the desired property from the collected results.
+
+### Relaxation
+
+The relaxation of a structure should be carried out before calculating all other properties:
+```bash
+dpgen autotest make equi.json 
+dpgen autotest run relax.json machine.json
+dpgen autotest post equi.json 
+```
+If, for some reason, the main program terminated at stage `run`, one can easily restart with the same command.
+`relax.json` is the parameter file. An example for `deepmd` relaxation is given as:
+```json
+{
+	"structures":	"confs/mp-*",
+	"interaction": {
+		"type":		"deepmd",
+		"model":	"frozen_model.pb",
+                "type_map":     {"Al": 0, "Mg": 1}
+	},
+	"relaxation": {
+	}
+}
+```
+
+where the key `structures` provides the structures to relax. `interaction` is provided with `deepmd`, and other options are `vasp`, `eam`, `meam`...
+
+Yuzhi:
+
+1. We should notice that the `interaction` here should always be considered as a unified abstract class, which means that we should avoid repeating identifing which interaction we're using in the main code.
+2. The structures here should always considered as a list, and the wildcard should be supported by using `glob`. Before all calculations , there is a stage where we generate the configurations.
+
+The outputs of the relaxation are stored in the `mp-*/00.relaxation` directory.
+```bash
+ls mp-*
+mp-1/relaxation  mp-2/relaxation  mp-3/relaxation
+```
+
+### Other properties
+
+Other properties can be computed in parallel:
+```bash
+dpgen autotest make properties.json 
+dpgen autotest run properties.json machine.json
+dpgen autotest post properties.json 
+```
+where an example of `properties.json` is given by
+```json
+{
+	"structures":	"confs/mp-*",
+	"interaction": {
+		"type":		"vasp",
+		"incar":	"vasp_input/INCAR",
+		"potcar_prefix":"vasp_input",
+		"potcars":	{"Al": "POTCAR.al", "Mg": "POTCAR.mg"}
+	},
+	"properties": [
+		{
+                        "type":         "eos",
+			"vol_start":	10,
+			"vol_end":	30,
+			"vol_step":	0.5
+		},
+		{
+                        "type":         "elastic",
+			"norm_deform":	2e-2,
+			"shear_deform": 5e-2
+		}
+        ]
+}
+```
+
+
+The `dpgen` packed all `eos` and `elastic` task and sends them to corresponding computational resources defined in `machine.json`. The outputs of a property, taking `eos` for example, are stored in
+```bash
+ls mp-*/ | grep eos
+mp-1/eos_00  mp-2/eos_00  mp-3/eos_00
+```
+where `00` are suffix of the task.
+
+### Refine the calculation of a property
+
+Some times we want to refine the calculation of a property from previous results. For example, when higher convergence criteria `EDIFF` and `EDIFFG` are necessary, and the new VASP calculation is desired to start from the previous output configration, rather than starting from scratch. 
+```bash
+dpgen autotest make refine.json 
+dpgen autotest run refine.json machine.json
+```
+with `refine.json`
+```json
+{
+	"properties": {
+		"eos" : {
+			"init_from_suffix":	"00",
+                        "output_suffix":        "01",
+			"vol_start":	10,
+			"vol_end":	30,
+			"vol_step":	0.5
+		}
+	}	
+}
+```
+
+
+
+### Configuration filter
+
+Some times the configurations automatically generated are problematic. For example, the distance between the interstitial atom and the lattic is too small, then these configurations should be filtered out. One can set filters of configurations by
+```json
+{
+	"properties": {
+		"intersitital" : {
+			"supercell":	[3,3,3],
+			"insert_atom":	["Al"],
+			"conf_filters": [
+				{  "min_dist": 2 }
+			] 
+		}
+	}	
+}
+```

doc/Autotest/EOS-get-started-and-input-examples.md

@@ -0,0 +1,26 @@
+Equation of State (EOS) here calculates the energies of the most stable structures as a function of volume. Users can refer to Figure 4 of the [dpgen JCPC paper](https://www.sciencedirect.com/science/article/pii/S001046552030045X?via%3Dihub) for more information of EOS.
+
+#### An example of the input file for EOS by VASP:
+
+```json
+{
+	"structures":	["confs/mp-*","confs/std-*","confs/test-*"],
+	"interaction": {
+		"type":		"vasp",
+		"incar":	"vasp_input/INCAR",
+                "potcar_prefix":"vasp_input",
+		"potcars":	{"Al": "POTCAR.al", "Mg": "POTCAR.mg"}
+	},
+	"properties": [
+		{
+                        "type":         "eos",
+			"vol_start":	10,
+			"vol_end":	30,
+			"vol_step":	0.5,
+                        "change_box":   true
+		}
+        ]
+}
+```
+
+`vol_start` is the starting volume per atom in Å^3/atom, `vol_step` is the increasing step of volume and the biggest volume is smaller than `vol_end`. In the above example, 40 tasks would be generated as `task.000000` to `task.000039` with the volume `10.00, 10.50, 11.00, ..., 29.50` Å^3/atom, respectively.

doc/Autotest/EOS-make.md

@@ -0,0 +1,110 @@
+**Step 1.** Before `make` in EOS, the equilibrium configuration `CONTCAR` must be present in `confs/mp-*/relaxation`. 
+
+**Step 2.** For the input example in the previous section, when we do `make`, 40 tasks would be generated as `confs/mp-*/eos_00/task.000000, confs/mp-*/eos_00/task.000001, ... , confs/mp-*/eos_00/task.000039`. The suffix `00` is used  for possible `refine` later. 
+
+**Step 3.** If the task directory, for example `confs/mp-*/eos_00/task.000000` is not empty, the old input files in it including `INCAR`, `POSCAR`, `POTCAR`, `conf.lmp`, `in.lammps` would be deleted. 
+
+**Step 4.** In each task directory, `POSCAR.orig` would link to `confs/mp-*/relaxation/CONTCAR`. Then the `scale` parameter can be calculated as:
+
+```txt
+scale = (vol_current / vol_equi) ** (1. / 3.)
+```
+
+`vol_current` is the corresponding volume per atom of the current task and `vol_equi` is the volume per atom of the equilibrium configuration. Then the `poscar_scale` function in `dpgen.auto_test.lib.vasp` module would help to generate `POSCAR` file with `vol_current` in `confs/mp-*/eos_00/task.[0-9]*[0-9]`.
+
+**Step 5.** According to the task type, the input file including `INCAR`, `POTCAR` or `conf.lmp`, `in.lammps` would be written in every `confs/mp-*/eos_00/task.[0-9]*[0-9]`. 
+
+For EOS calculations by VASP, if `change_box` is `True`, `ISIF` in VASP would be 4, else `ISIF` would be 2. The default value of `change_box` is `True`. For further information of the use of `ISIF` in VASP, we refer users to [ISIF command](https://www.vasp.at/wiki/index.php/ISIF).
+
+For EOS calculations by LAMMPS, when `change_box` is `True`, an example of `in.lammps` for AlMg is given as below and the `scale` parameter in line 5 is calculated by the equation above.
+
+```txt
+clear
+variable        GPa2bar	equal 1e4
+variable        B0	equal 70
+variable        bp	equal 0
+variable	xx	equal scale
+variable        yeta	equal 1.5*(${bp}-1)
+variable        Px0	equal 3*${B0}*(1-${xx})/${xx}^2*exp(${yeta}*(1-${xx}))
+variable        Px	equal ${Px0}*${GPa2bar}
+units           metal
+dimension       3
+boundary	p p p
+atom_style	atomic
+box             tilt large
+read_data       conf.lmp
+mass            1 1
+mass            2 1
+neigh_modify    every 1 delay 0 check no
+pair_style      deepmd frozen_model.pb
+pair_coeff
+compute         mype all pe
+thermo          100
+thermo_style    custom step pe pxx pyy pzz pxy pxz pyz lx ly lz vol c_mype
+dump            1 all custom 100 dump.relax id type xs ys zs fx fy fz
+min_style       cg
+fix             1 all box/relax iso ${Px}
+minimize        1.000000e-12 1.000000e-06 5000 500000
+fix             1 all box/relax aniso ${Px}
+minimize        1.000000e-12 1.000000e-06 5000 500000
+variable        N equal count(all)
+variable        V equal vol
+variable        E equal "c_mype"
+variable        Pxx equal pxx
+variable        Pyy equal pyy
+variable        Pzz equal pzz
+variable        Pxy equal pxy
+variable        Pxz equal pxz
+variable        Pyz equal pyz
+variable        Epa equal ${E}/${N}
+variable        Vpa equal ${V}/${N}
+print "All done"
+print "Total number of atoms  = ${N}"
+print "Relax at Press         = ${Px} Bar"
+print "Final energy per atoms = ${Epa} eV"
+print "Final volume per atoms = ${Vpa} A^3"
+print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${Pyz}"
+```
+
+when `change_box` is `False`, an example of `in.lammps` for AlMg is given as:
+```txt
+clear
+units 	metal
+dimension	3
+boundary	p	p    p
+atom_style	atomic
+box         tilt large
+read_data   conf.lmp
+mass            1 1
+mass            2 1
+neigh_modify    every 1 delay 0 check no
+pair_style deepmd frozen_model.pb
+pair_coeff
+compute         mype all pe
+thermo          100
+thermo_style    custom step pe pxx pyy pzz pxy pxz pyz lx ly lz vol c_mype
+dump            1 all custom 100 dump.relax id type xs ys zs fx fy fz
+min_style       cg
+minimize        1.000000e-12 1.000000e-06 5000 500000
+variable        N equal count(all)
+variable        V equal vol
+variable        E equal "c_mype"
+variable        tmplx equal lx
+variable        tmply equal ly
+variable        Pxx equal pxx
+variable        Pyy equal pyy
+variable        Pzz equal pzz
+variable        Pxy equal pxy
+variable        Pxz equal pxz
+variable        Pyz equal pyz
+variable        Epa equal ${E}/${N}
+variable        Vpa equal ${V}/${N}
+variable        AA equal (${tmplx}*${tmply})
+print "All done"
+print "Total number of atoms = ${N}"
+print "Final energy per atoms = ${Epa}"
+print "Final volume per atoms = ${Vpa}"
+print "Final Base area = ${AA}"
+print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${Pyz}"
+```
+

doc/Autotest/EOS-post.md

@@ -0,0 +1,33 @@
+The post processing of EOS would go to every directory in `confs/mp-*/eos_00` and do the post processing. Let's suppose we are now in `confs/mp-100/eos_00` and there are `task.000000, task.000001,..., task.000039` in this directory. By reading `inter.json` file in every task directory, the task type can be determined and the energy and force information of every task can further be obtained. By appending the `dict` of energy and force into a list, an example of the list with 1 atom is given as:
+```txt
+[
+    {"energy": E1, "force": [fx1, fy1, fz1]},
+    {"energy": E2, "force": [fx2, fy2, fz2]},
+    ...
+    {"energy": E40, "force": [fx40, fy40, fz40]}
+]
+```
+Then the volume can be calculated from the task id and the corresponding energy can be obtained from the list above. Finally, there would be `result.json` in json format and `result.out` in txt format in `confs/mp-100/eos_00` containing the EOS results.
+
+An example of `result.json` is give as:
+```txt
+{
+    10.00: -3.0245,
+    10.50: -3.0216,
+         ...
+    29.50: -7.9740
+}
+```
+
+An example of `result.out` is given below:
+
+```txt
+conf_dir: confs/mp-100/eos_00
+ VpA(A^3)  EpA(eV)
+ 10.000   -3.0245
+ 10.500   -3.0216
+  ...        ...
+ 29.500   -7.9740
+```
+
+

doc/Autotest/EOS-run.md

@@ -0,0 +1,3 @@
+The work path of each task should be in the form like `confs/mp-*/eos_00` and all task is in the form like `confs/mp-*/eos_00/task.[0-9]*[0-9]`.
+
+When we dispatch tasks, we would go through every individual work path in the list `confs/mp-*/eos_00`, and then submit `task.[0-9]*[0-9]` in each work path.

doc/Autotest/Elastic-get-started-and-input-examples.md

@@ -0,0 +1,28 @@
+Here we calculate the mechanical properties which include elastic constants (C11 to C66), bulk modulus Bv, shear modulus Gv, Youngs modulus Ev, and Poission ratio Uv of a certain crystal structure.
+
+#### An example of the input file for Elastic by deepmd:
+
+```json
+{
+	"structures":	["confs/mp-*","confs/std-*","confs/test-*"],
+	"interaction": {
+		"type":		"deepmd",
+                "model":        "frozen_model.pb",
+		"type_map":	{"Al": 0, "Mg": 1}
+	},
+	"properties": [
+            {
+                "type":         "elastic",
+                "norm_deform":	2e-2,
+	       "shear_deform":  5e-2
+	    }
+        ]
+}
+```
+
+Here the default values of `norm_deform` and `shear_deform` are **2e-3** and **5e-3**, respectively. A list of `norm_strains` and `shear_strains` would be generated as below:
+
+```bash
+[-norm_def, -0.5 * norm_def, 0.5 * norm_def, norm_def]
+[-shear_def, -0.5 * shear_def, 0.5 * shear_def, shear_def]
+```