add and run pre-commit to format codes

.github/ISSUE_TEMPLATE/request-for-help.md

@@ -13,7 +13,7 @@ Before asking questions, you can
 search the previous issues or discussions
 check [Manual](https://github.com/deepmodeling/dpgen).
 
-Please **do not** post requests for help (e.g. with installing or using dpgen) here. 
+Please **do not** post requests for help (e.g. with installing or using dpgen) here.
 Instead go to [discussions](https://github.com/deepmodeling/dpgen/discussions).
 
 This issue tracker is for tracking dpgen development related issues only.

.gitignore

@@ -42,4 +42,3 @@ dbconfig.json
 _build
 tests/generator/calypso_test_path
 doc/api/
-

.pre-commit-config.yaml

@@ -0,0 +1,27 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+    # there are many log files in tests
+    # TODO: seperate py files and log files
+    -   id: trailing-whitespace
+        exclude: "^tests/.*$"
+    -   id: end-of-file-fixer
+        exclude: "^tests/.*$"
+    -   id: check-yaml
+        exclude: "^conda/.*$"
+    #-   id: check-json
+    -   id: check-added-large-files
+    -   id: check-merge-conflict
+    -   id: check-symlinks
+        exclude: "^tests/tools/run_report_test_output/.*$"
+    -   id: check-toml
+# Python
+-   repo: https://github.com/psf/black
+    rev: 22.12.0
+    hooks:
+    -   id: black-jupyter
+ci:
+  autoupdate_branch: devel

conda/conda_build_config.yaml

@@ -1,5 +1,5 @@
-channel_sources: 
+channel_sources:
   - defaults
   - conda-forge
-channel_targets: 
+channel_targets:
   - deepmodeling

conda/construct.yaml

@@ -11,7 +11,7 @@ channels:
 specs:
   - python 3.8
   - pip
-  - dpgen {{ version }} 
+  - dpgen {{ version }}
 
 ignore_duplicate_files: True
 

doc/autotest/Auto-test.md

@@ -30,8 +30,8 @@ where the key `structures` provides the structures to relax. `interaction` is pr
 ### Task type
 There are now six task types implemented in the package: `vasp`, `abacus`, `deepmd`, `meam`, `eam_fs`, and `eam_alloy`. An `inter.json` file in json format containing the interaction parameters will be written in the directory of each task after `make`. We give input examples of the `interaction` part for each type below:
 
-**VASP**: 
-    
+**VASP**:
+
 The default of `potcar_prefix` is "".
 ```json
 	"interaction": {
@@ -41,8 +41,8 @@ The default of `potcar_prefix` is "".
 		"potcars":	{"Al": "POTCAR.al", "Mg": "POTCAR.mg"}
 	}
 ```
-**ABACUS**: 
-    
+**ABACUS**:
+
 The default of `potcar_prefix` is "". The path of potcars/orb_files/deepks_desc is `potcar_prefix` + `potcars`/`orb_files`/`deepks_desc`.
 ```json
 	"interaction": {
@@ -62,7 +62,7 @@ The default of `potcar_prefix` is "". The path of potcars/orb_files/deepks_desc
 ```json
 	"interaction": {
 		"type":		 "deepmd",
-		"model":	 "frozen_model.pb", 
+		"model":	 "frozen_model.pb",
 		"type_map":      {"Al": 0, "Mg": 1}
 	}
 ```
@@ -81,8 +81,8 @@ Please make sure the [USER-MEAMC package](https://lammps.sandia.gov/doc/Packages
 Please make sure the [MANYBODY package](https://lammps.sandia.gov/doc/Packages_details.html#pkg-manybody) has already been installed in LAMMPS
 ```json
 	"interaction": {
-		"type":		 "eam_fs (eam_alloy)", 
-		"model":	 "AlMg.eam.fs (AlMg.eam.alloy)", 
+		"type":		 "eam_fs (eam_alloy)",
+		"model":	 "AlMg.eam.fs (AlMg.eam.alloy)",
 		"type_map":      {"Al": 1, "Mg": 2}
 	}
 ```
@@ -96,9 +96,9 @@ Now the supported property types are `eos`, `elastic`, `vacancy`, `interstitial`
 There are three operations in auto test package, namely `make`, `run`, and `post`. Here we take `eos` property as an example for property type.
 
 ### Make
-The `INCAR`, `POSCAR`, `POTCAR` input files for VASP or `in.lammps`, `conf.lmp`, and the interatomic potential files for LAMMPS will be generated in the directory `confs/mp-*/relaxation/relax_task` for relaxation or `confs/mp-*/eos_00/task.[0-9]*[0-9]` for EOS. The `machine.json` file is not needed for `make`. Example: 
+The `INCAR`, `POSCAR`, `POTCAR` input files for VASP or `in.lammps`, `conf.lmp`, and the interatomic potential files for LAMMPS will be generated in the directory `confs/mp-*/relaxation/relax_task` for relaxation or `confs/mp-*/eos_00/task.[0-9]*[0-9]` for EOS. The `machine.json` file is not needed for `make`. Example:
 ```bash
-dpgen autotest make relaxation.json 
+dpgen autotest make relaxation.json
 ```
 
 ### Run
@@ -110,5 +110,5 @@ dpgen autotest run relaxation.json machine.json
 ### Post
 The post process of calculation results would be performed. `result.json` in json format will be generated in `confs/mp-*/relaxation/relax_task` for relaxation and `result.json` in json format and `result.out` in txt format in `confs/mp-*/eos_00` for EOS. The `machine.json` file is also not needed for `post`. Example:
 ```bash
-dpgen autotest post relaxation.json 
+dpgen autotest post relaxation.json
 ```

doc/autotest/index.rst

@@ -9,13 +9,13 @@ Auto test
    :caption: Guidelines
 
    Auto-test
-  
+
 .. _Main-components::
 
 .. toctree::
    :maxdepth: 2
    :caption: Main components
-   
+
    Task-type
    Property-type
    Make-run-and-post
@@ -25,7 +25,7 @@ Auto test
 .. toctree::
    :maxdepth: 2
    :caption: Structure relaxation
-   
+
    relaxation/index.rst
 
 .. _Property::
@@ -41,13 +41,13 @@ Auto test
 .. toctree::
    :maxdepth: 2
    :caption: Refine
-   
+
    refine/index.rst
 
 .. _Reproduce::
 
 .. toctree::
    :maxdepth: 2
    :caption: Reproduce
-   
+
    reproduce/index.rst

doc/autotest/property/Property-get-started-and-input-examples.md

@@ -61,7 +61,7 @@ Key words | data structure | example | description
 **type** | String | "eos" | property type
 skip | Boolean | true | whether to skip current property or not
 start_confs_path | String | "../vasp/confs" | start from the equilibrium configuration in other path only for the current property type
-cal_setting["input_prop"] | String | "lammps_input/lammps_high" |input commands file 
+cal_setting["input_prop"] | String | "lammps_input/lammps_high" |input commands file
 cal_setting["overwrite_interaction"] | Dict | | overwrite the interaction in the `interaction` part only for the current property type
 
 other parameters in `cal_setting` and `cal_type` in `relaxation` also apply in `property`.
@@ -116,4 +116,4 @@ Key words | data structure | example | description
 supercell_size | List of Int | [1,1,10] | the supercell to be constructed, default = [1,1,5]
 min_vacuum_size | Int or Float | 10 | minimum size of vacuum width, default = 20
 add_fix | List of String | ['true','true','false'] | whether to fix atoms in the direction, default = ['true','true','false'] (standard method)
-n_steps | Int | 20 | Number of points for gamma-line calculation, default = 10
+n_steps | Int | 20 | Number of points for gamma-line calculation, default = 10

doc/autotest/property/Property-post.md

@@ -4,4 +4,4 @@ Use command
 ```bash
 dpgen autotest post property.json
 ```
-to post results as `result.json` and `result.out` in each property's path.
+to post results as `result.json` and `result.out` in each property's path.

doc/autotest/property/index.rst

@@ -9,4 +9,4 @@ Property
    Property-make
    Property-run
    Property-post
-   properties/index.rst
+   properties/index.rst

doc/autotest/property/properties/EOS-make.md

@@ -1,10 +1,10 @@
 ## EOS make
 
-**Step 1.** Before `make` in EOS, the equilibrium configuration `CONTCAR` must be present in `confs/mp-*/relaxation`. 
+**Step 1.** Before `make` in EOS, the equilibrium configuration `CONTCAR` must be present in `confs/mp-*/relaxation`.
 
-**Step 2.** For the input example in the previous section, when we do `make`, 40 tasks would be generated as `confs/mp-*/eos_00/task.000000, confs/mp-*/eos_00/task.000001, ... , confs/mp-*/eos_00/task.000039`. The suffix `00` is used  for possible `refine` later. 
+**Step 2.** For the input example in the previous section, when we do `make`, 40 tasks would be generated as `confs/mp-*/eos_00/task.000000, confs/mp-*/eos_00/task.000001, ... , confs/mp-*/eos_00/task.000039`. The suffix `00` is used  for possible `refine` later.
 
-**Step 3.** If the task directory, for example `confs/mp-*/eos_00/task.000000` is not empty, the old input files in it including `INCAR`, `POSCAR`, `POTCAR`, `conf.lmp`, `in.lammps` would be deleted. 
+**Step 3.** If the task directory, for example `confs/mp-*/eos_00/task.000000` is not empty, the old input files in it including `INCAR`, `POSCAR`, `POTCAR`, `conf.lmp`, `in.lammps` would be deleted.
 
 **Step 4.** In each task directory, `POSCAR.orig` would link to `confs/mp-*/relaxation/CONTCAR`. Then the `scale` parameter can be calculated as:
 
@@ -14,4 +14,4 @@ scale = (vol_current / vol_equi) ** (1. / 3.)
 
 `vol_current` is the corresponding volume per atom of the current task and `vol_equi` is the volume per atom of the equilibrium configuration. Then the `poscar_scale` function in `dpgen.auto_test.lib.vasp` module would help to generate `POSCAR` file with `vol_current` in `confs/mp-*/eos_00/task.[0-9]*[0-9]`.
 
-**Step 5.** According to the task type, the input file including `INCAR`, `POTCAR` or `conf.lmp`, `in.lammps` would be written in every `confs/mp-*/eos_00/task.[0-9]*[0-9]`. 
+**Step 5.** According to the task type, the input file including `INCAR`, `POTCAR` or `conf.lmp`, `in.lammps` would be written in every `confs/mp-*/eos_00/task.[0-9]*[0-9]`.

doc/autotest/property/properties/EOS-post.md

@@ -31,5 +31,3 @@ onf_dir: /root/auto_test_example/deepmd/confs/std-fcc/eos_00
    ...      ...
  17.935   -3.7088
 ```
-
-

doc/autotest/property/properties/Interstitial-make.md

@@ -4,4 +4,4 @@
 
 **Step 2.** If `refine` is `True`, we do [refine process](../../refine/Refine-get-started-and-input-examples). If `reprod-opt` is `True` (the default is **False**), we do [reproduce process](../../reproduce/Reproduce-get-started-and-input-examples). Else, the vacancy structure (`POSCAR`) and supercell information (`supercell.out`) are written in the task directory, for example, in `confs/mp-*/interstitial_00/task.000000` with the check and possible removing of the old input files like before.
 
-**Step 3.** In `interstitial` by VASP, `ISIF = 3`. In `interstitial` by LAMMPS, the same `in.lammps` as that in [EOS (change_box is True)](./EOS-make) would be generated with `scale` set to one. 
+**Step 3.** In `interstitial` by VASP, `ISIF = 3`. In `interstitial` by LAMMPS, the same `in.lammps` as that in [EOS (change_box is True)](./EOS-make) would be generated with `scale` set to one.

doc/autotest/property/properties/Interstitial-post.md

@@ -1,6 +1,6 @@
 ## Interstitial post
 
-For `Interstitial`, we need to calculate the energy difference between a crystal structure with and without atom added in. 
+For `Interstitial`, we need to calculate the energy difference between a crystal structure with and without atom added in.
 The examples of the output files `result.json` in json format and `result.out` in txt format are given below.
 
 #### result.json

doc/autotest/property/properties/Surface-get-started-and-input-examples.md

@@ -1,6 +1,6 @@
 ## Surface get started and input examples
 
-`Surface` calculates the surface energy. We need to give the information of `min_slab_size`, `min_vacuum_size`, `max_miller` (default value is 2), and `pert_xz` which means perturbations in xz and will help work around vasp bug. 
+`Surface` calculates the surface energy. We need to give the information of `min_slab_size`, `min_vacuum_size`, `max_miller` (default value is 2), and `pert_xz` which means perturbations in xz and will help work around vasp bug.
 
 #### An example of the input file for Surface by deepmd:
 
@@ -18,7 +18,7 @@
                 "min_slab_size":  10,
                 "min_vacuum_size":11,
                 "max_miller":     2,
-                "cal_type":       "static" 
+                "cal_type":       "static"
 	    }
         ]
 }

doc/autotest/property/properties/Surface-make.md

@@ -1,5 +1,5 @@
 ## Surface make
 
-**Step 1.** Based on the equilibrium configuration, `generate_all_slabs` module in [pymatgen.core.surface](https://pymatgen.org/pymatgen.core.surface.html) would help to generate surface structure list with using `max_miller`, `min_slab_size`, and `min_vacuum_size` parameters. 
+**Step 1.** Based on the equilibrium configuration, `generate_all_slabs` module in [pymatgen.core.surface](https://pymatgen.org/pymatgen.core.surface.html) would help to generate surface structure list with using `max_miller`, `min_slab_size`, and `min_vacuum_size` parameters.
 
 **Step 2.** If `refine` is True, we do [refine process](../../refine/Refine-get-started-and-input-examples). If `reprod-opt` is True (the default is False), we do [reproduce process](../../reproduce/Reproduce-get-started-and-input-examples). Otherwise, the surface structure (`POSCAR`) with perturbations in xz and miller index information (`miller.out`) are written in the task directory, for example, in `confs/mp-*/interstitial_00/task.000000` with the check and possible removing of the old input files like before.

doc/autotest/property/properties/Vacancy-make.md

@@ -1,7 +1,7 @@
 ## Vacancy make
 
-**Step 1.** The `VacancyGenerator` module in [pymatgen.analysis.defects.generators](https://pymatgen.org/pymatgen.analysis.defects.generators.html) is used to generate a set of structures with vacancy. 
+**Step 1.** The `VacancyGenerator` module in [pymatgen.analysis.defects.generators](https://pymatgen.org/pymatgen.analysis.defects.generators.html) is used to generate a set of structures with vacancy.
 
 **Step 2.** If there are `init_from_suffix` and `output_suffix` parameter in the `properties` part, the [refine process](../../refine/Refine-get-started-and-input-examples) follows. If reproduce is evoked, the [reproduce process](../../reproduce/Reproduce-get-started-and-input-examples) follows. Otherwise, the vacancy structure (`POSCAR`) and supercell information (`supercell.out`) are written in the task directory, for example, in `confs/mp-*/vacancy_00/task.000000` with the check and possible removing of the old input files like before.
 
-**Step 3.** When doing `vacancy` by VASP, `ISIF = 3`. When doing `vacancy` by LAMMPS, the same `in.lammps` as that in [EOS (change_box is True)](./EOS-make) would be generated with `scale` set to one. 
+**Step 3.** When doing `vacancy` by VASP, `ISIF = 3`. When doing `vacancy` by LAMMPS, the same `in.lammps` as that in [EOS (change_box is True)](./EOS-make) would be generated with `scale` set to one.

doc/autotest/property/properties/Vacancy-post.md

@@ -1,6 +1,6 @@
 ## Vacancy post
 
-For `Vacancy`, we need to calculate the energy difference between a crystal structure with and without a vacancy. 
+For `Vacancy`, we need to calculate the energy difference between a crystal structure with and without a vacancy.
 The examples of the output files `result.json` in json format and `result.out` in txt format are given below.
 
 #### result.json

doc/autotest/property/properties/index.rst

@@ -24,4 +24,4 @@ Properties
    Surface-get-started-and-input-examples
    Surface-make
    Surface-run
-   Surface-post
+   Surface-post

doc/autotest/refine/Refine-get-started-and-input-examples.md

@@ -1,6 +1,6 @@
 ## Refine get started and input examples
 
-Sometimes we want to refine the calculation of a property from previous results. For example, when higher convergence criteria `EDIFF` and `EDIFFG` are necessary in VASP, the new VASP calculation is desired to start from the previous output configuration, rather than starting from scratch. 
+Sometimes we want to refine the calculation of a property from previous results. For example, when higher convergence criteria `EDIFF` and `EDIFFG` are necessary in VASP, the new VASP calculation is desired to start from the previous output configuration, rather than starting from scratch.
 
 
 An example of the input file `refine.json` is given below:

doc/autotest/refine/index.rst

@@ -4,8 +4,8 @@ Refine
 
 .. toctree::
    :maxdepth: 2
-   
+
    Refine-get-started-and-input-examples
    Refine-make
    Refine-run
-   Refine-post
+   Refine-post

doc/autotest/relaxation/Relaxation-get-started-and-input-examples.md

@@ -1,8 +1,8 @@
 ## Relaxation get started and input examples
 
-The relaxation of a structure should be carried out before calculating all other properties. 
+The relaxation of a structure should be carried out before calculating all other properties.
 
-First, we need input parameter file and we name it `relax.json` here. All the relaxation calculations should be taken either by `VASP`, `ABACUS`, or `LAMMPS`. Here are two input examples for `VASP` and `LAMMPS` respectively. 
+First, we need input parameter file and we name it `relax.json` here. All the relaxation calculations should be taken either by `VASP`, `ABACUS`, or `LAMMPS`. Here are two input examples for `VASP` and `LAMMPS` respectively.
 
 An example of the input file for relaxation by VASP:
 
@@ -81,5 +81,3 @@ maxiter | Int | 5000 | max iterations of minimizer
 maxeval | Int | 500000 | max number of force/energy evaluations
 
 For LAMMPS relaxation and all the property calculations, **package will help to generate `in.lammps` file for user automatically** according to the property type. We can also make the final changes in the `minimize` setting (`minimize etol ftol maxiter maxeval`) in `in.lammps`. In addition, users can apply the input file for lammps commands in the `interaction` part. For further information of the LAMMPS relaxation, we refer users to [minimize command](https://lammps.sandia.gov/doc/minimize.html).
-
-

doc/autotest/relaxation/Relaxation-make.md

@@ -96,7 +96,3 @@ print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${P
 If user provides lammps input command file `in.lammps`, the `thermo_style` and `dump` commands should be the same as the above file.
 
 **interatomic potential model**: the `frozen_model.pb` in `confs/mp-*/relaxation` would link to the `frozen_model.pb` file given in the input.
-
-
-
-

doc/autotest/relaxation/Relaxation-run.md

@@ -1,6 +1,6 @@
 ## Relaxation run
 
-The work path of each task should be in the form like `confs/mp-*/relaxation` and all task is in the form like `confs/mp-*/relaxation/relax_task`. 
+The work path of each task should be in the form like `confs/mp-*/relaxation` and all task is in the form like `confs/mp-*/relaxation/relax_task`.
 
 The `machine.json` file should be applied in this process and the machine parameters (eg. GPU or CPU) are determined according to the task type (VASP or LAMMPS). Then in each work path, the corresponding tasks would be submitted and the results would be sent back through [make_dispatcher](https://github.com/deepmodeling/dpgen/blob/devel/dpgen/dispatcher/Dispatcher.py).
 

doc/autotest/reproduce/Reproduce-get-started-and-input-examples.md

@@ -22,8 +22,8 @@ Sometimes we want to reproduce the initial results with the same configurations
 }
 ```
 
-`reproduce` denotes whether to do `reproduce` or not and the default value is False. 
+`reproduce` denotes whether to do `reproduce` or not and the default value is False.
 
-`init_data_path` is the path of VASP or LAMMPS initial data to be reproduced. `init_from_suffix` is the suffix of the initial data and the default value is "00". In this case, the VASP Interstitial results are stored in `../vasp/confs/std-*/interstitial_00` and the reproduced Interstitial results would be in `deepmd/confs/std-*/interstitial_reprod`. 
+`init_data_path` is the path of VASP or LAMMPS initial data to be reproduced. `init_from_suffix` is the suffix of the initial data and the default value is "00". In this case, the VASP Interstitial results are stored in `../vasp/confs/std-*/interstitial_00` and the reproduced Interstitial results would be in `deepmd/confs/std-*/interstitial_reprod`.
 
-`reprod_last_frame` denotes if only the last frame is used in reproduce. The default value is True for eos and surface, but is False for vacancy and interstitial.
+`reprod_last_frame` denotes if only the last frame is used in reproduce. The default value is True for eos and surface, but is False for vacancy and interstitial.

doc/autotest/reproduce/index.rst

@@ -4,8 +4,8 @@ Reproduce
 
 .. toctree::
    :maxdepth: 2
-   
+
    Reproduce-get-started-and-input-examples
    Reproduce-make
    Reproduce-run
-   Reproduce-post
+   Reproduce-post