Better user interface and consistency check through out env/gp/kernel modules

.gitignore

@@ -20,6 +20,9 @@ pwscf.save/
 /tests/test_ase/__pycache__/
 /tests/test_outputs/*
 test_files/qe_output_*.out
+/tests/*log*
+/tests/*.pickle
+/tests/*.npy
 /tests/*.xyz
 /tests/*.gp
 /tests/*-bak

.travis.yml

@@ -14,17 +14,19 @@ before_install:
     - pwd
     - wget http://folk.uio.no/anjohan/lmp
     - "wget https://github.com/cp2k/cp2k/releases/download/\
-      v6.1.0/cp2k-6.1-Linux-x86_64.sopt"
+      v7.1.0/cp2k-7.1-Linux-x86_64.sopt"
     - chmod u+x lmp
-    - chmod u+x cp2k-6.1-Linux-x86_64.sopt
+    - chmod u+x cp2k-7.1-Linux-x86_64.sopt
     - pip install -r requirements.txt
 
 script:
     - pwd
     - cd tests
-    - PWSCF_COMMAND=pw.x CP2K_COMMAND=../cp2k-6.1-Linux-x86_64.sopt 
+    - ls test_files
+    - PWSCF_COMMAND=pw.x
       lmp=$(pwd)/../lmp
-      pytest --show-capture=all -vv --durations=0 --cov=../flare/
+      CP2K_COMMAND=../cp2k-7.1-Linux-x86_64.sopt
+      pytest -vv --durations=0 --cov=../flare/
     - coverage xml
 
 after_success:

docs/source/conf.py

@@ -47,7 +47,8 @@
     'sphinx.ext.autodoc',
     'sphinx.ext.imgmath',
     'sphinx_rtd_theme',
-    'sphinx.ext.napoleon'
+    'sphinx.ext.napoleon',
+    'nbsphinx'
 ]
 napoleon_use_param = False
 # Add any paths that contain templates here, relative to this directory.

docs/source/flare/ase/ase.rst

@@ -3,11 +3,9 @@ ASE Interface
 
 We provide an interface to do the OTF training coupling with ASE. 
 Including wrapping the FLARE's GaussianProcess and MappedGaussianProcess into an ASE calculator: :class:`FLARE_Calculator`,
-the on-the-fly training module coupled with ASE molecular dynamics engines :module:`otf_md.py`
 
 .. toctree::
    :maxdepth: 2
 
    calculator
    otf
-   otf_md

docs/source/flare/utils/env_getarray.rst

@@ -0,0 +1,5 @@
+Construct Atomic Environment
+===================
+
+.. automodule:: flare.utils.env_getarray
+    :members:

docs/source/flare/utils/flare_io.rst

@@ -0,0 +1,6 @@
+I/O for trajectories
+===================
+
+.. automodule:: flare.utils.flare_io
+    :members:
+

docs/source/flare/utils/mask_helper.rst

@@ -1,5 +1,5 @@
 Advanced Hyperparameters Set Up
 ===================
 
-.. automodule:: flare.utils.mask_helper
+.. automodule:: flare.utils.parameter_helper
     :members:

docs/source/flare/utils/utils.rst

@@ -7,3 +7,6 @@ Utility
    element_coder
    learner
    mask_helper
+   env_getarray
+   md_helper
+   flare_io

docs/source/tutorials/after_training.ipynb

@@ -0,0 +1,265 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# After Training\n",
+    "\n",
+    "After the on-the-fly training is complete, we can play with the force field we obtained. \n",
+    "We are going to do the following things:\n",
+    "\n",
+    "1. Parse the on-the-fly training trajectory to collect training data\n",
+    "2. Reconstruct the GP model from the training trajectory\n",
+    "3. Build up Mapped GP (MGP) for accelerated force field, and save coefficient file for LAMMPS\n",
+    "4. Use LAMMPS to run fast simulation using MGP pair style\n",
+    "\n",
+    "## Parse OTF log file\n",
+    "\n",
+    "After the on-the-fly training is complete, we have a log file and can use the `otf_parser` module to parse the trajectory. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "from flare import otf_parser\n",
+    "\n",
+    "logdir = '../../../tests/test_files'\n",
+    "file_name = f'{logdir}/AgI_snippet.out'\n",
+    "hyp_no = 2 # use the hyperparameters from the 2nd training step\n",
+    "otf_object = otf_parser.OtfAnalysis(file_name)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Construct GP model from log file\n",
+    "\n",
+    "We can reconstruct GP model from the parsed log file (the on-the-fly training trajectory). Here we build up the GP model with 2+3 body kernel from the on-the-fly log file. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Final name of the gp instance is default_gp_2\n"
+     ]
+    }
+   ],
+   "source": [
+    "gp_model = otf_object.make_gp(hyp_no=hyp_no)\n",
+    "gp_model.parallel = True\n",
+    "gp_model.hyp_labels = ['sig2', 'ls2', 'sig3', 'ls3', 'noise']\n",
+    "\n",
+    "# write model to a binary file\n",
+    "gp_model.write_model('AgI.gp', format='pickle')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The last step `write_model` is to write this GP model into a binary file, \n",
+    "so next time we can directly load the model from the pickle file as"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Final name of the gp instance is default_gp_2_2\n"
+     ]
+    }
+   ],
+   "source": [
+    "from flare.gp import GaussianProcess\n",
+    "\n",
+    "gp_model = GaussianProcess.from_file('AgI.gp.pickle')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Map the GP force field & Dump LAMMPS coefficient file\n",
+    "\n",
+    "To use the trained force field with accelerated version MGP, or in LAMMPS, we need to build MGP from GP model. \n",
+    "Since 2-body and 3-body are both included, we need to set up the number of grid points for 2-body and 3-body in `grid_params`.\n",
+    "We build up energy mapping, thus set `map_force=False`.\n",
+    "See [MGP tutorial](https://flare.readthedocs.io/en/latest/tutorials/mgp.html) for more explanation of the MGP settings."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from flare.mgp import MappedGaussianProcess\n",
+    "\n",
+    "grid_params = {'twobody':   {'grid_num': [64]}, \n",
+    "               'threebody': {'grid_num': [20, 20, 20]}}\n",
+    "\n",
+    "data = gp_model.training_statistics\n",
+    "lammps_location = 'AgI_Molten_15.txt'\n",
+    "\n",
+    "mgp_model = MappedGaussianProcess(grid_params, data['species'], \n",
+    "    map_force=False, lmp_file_name='AgI_Molten_15.txt', n_cpus=1)\n",
+    "mgp_model.build_map(gp_model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The coefficient file for LAMMPS mgp pair_style is automatically saved once the mapping is done. \n",
+    "Saved as `lmp_file_name`. \n",
+    "\n",
+    "## Run LAMMPS with MGP pair style\n",
+    "\n",
+    "With the above coefficient file, we can run LAMMPS simulation with the mgp pair style. \n",
+    "First download our mgp pair style files, compile your lammps executable with mgp pair style following our [instruction](https://flare.readthedocs.io/en/latest/tutorials/lammps.html).\n",
+    "\n",
+    "1. One way to use it is running `lmp_executable < in.lammps > log.lammps` \n",
+    "with the executable provided in our repository. \n",
+    "When creating the input file, please note to set\n",
+    "\n",
+    "```\n",
+    "newton off\n",
+    "pair_style mgp\n",
+    "pair_coeff * * <lmp_file_name> <chemical_symbols> yes/no yes/no\n",
+    "```\n",
+    "\n",
+    "An example is using coefficient file `AgI_Molten_15.txt` for AgI system, \n",
+    "with two-body (the 1st `yes`) together with three-body (the 2nd `yes`).\n",
+    "\n",
+    "```\n",
+    "pair_coeff * * AgI_Molten_15.txt Ag I yes yes\n",
+    "```\n",
+    "\n",
+    "**Note**: if you build force mapping (`map_force=True`) instead of energy mapping, please use\n",
+    "```\n",
+    "pair_style mgpf\n",
+    "```\n",
+    "\n",
+    "2. The third way is to use the ASE LAMMPS interface"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from flare.ase.calculator import FLARE_Calculator\n",
+    "\n",
+    "# get chemical symbols, masses etc.\n",
+    "species = gp_model.training_statistics['species']\n",
+    "specie_symbol_list = \" \".join(species)\n",
+    "masses=[f\"{i} {_Z_to_mass[_element_to_Z[species[i]]]}\" for i in range(len(species))]\n",
+    "\n",
+    "# set up input params\n",
+    "parameters = {'command': os.environ.get('lmp'), # set up executable for ASE\n",
+    "              'newton': 'off',\n",
+    "              'pair_style': 'mgp',\n",
+    "              'pair_coeff': [f'* * {lammps_location} {specie_symbol_list} yes yes'],\n",
+    "              'mass': masses}\n",
+    "files = [lammps_location]\n",
+    "\n",
+    "# create ASE calc\n",
+    "lmp_calc = LAMMPS(label=f'tmp_AgI', keep_tmp_files=True, tmp_dir='./tmp/',\n",
+    "        parameters=parameters, files=files, specorder=species)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "3. The second way to run LAMMPS is using our LAMMPS interface, please set the\n",
+    "environment variable `$lmp` to the executable."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from flare import struc\n",
+    "from flare.lammps import lammps_calculator\n",
+    "\n",
+    "# lmp coef file is automatically written now every time MGP is constructed\n",
+    "\n",
+    "# create test structure\n",
+    "species = otf_object.gp_species_list[-1]\n",
+    "positions = otf_object.position_list[-1]\n",
+    "forces = otf_object.force_list[-1]\n",
+    "otf_cell = otf_object.header['cell']\n",
+    "structure = struc.Structure(otf_cell, species, positions)\n",
+    "\n",
+    "atom_types = [1, 2]\n",
+    "atom_masses = [108, 127]\n",
+    "atom_species = [1, 2] * 27\n",
+    "\n",
+    "# create data file\n",
+    "data_file_name = 'tmp.data'\n",
+    "data_text = lammps_calculator.lammps_dat(structure, atom_types,\n",
+    "                                         atom_masses, atom_species)\n",
+    "lammps_calculator.write_text(data_file_name, data_text)\n",
+    "\n",
+    "# create lammps input\n",
+    "style_string = 'mgp'\n",
+    "coeff_string = '* * {} Ag I yes yes'.format(lammps_location)\n",
+    "lammps_executable = '$lmp'\n",
+    "dump_file_name = 'tmp.dump'\n",
+    "input_file_name = 'tmp.in'\n",
+    "output_file_name = 'tmp.out'\n",
+    "input_text = \\\n",
+    "    lammps_calculator.generic_lammps_input(data_file_name, style_string,\n",
+    "                                           coeff_string, dump_file_name)\n",
+    "lammps_calculator.write_text(input_file_name, input_text)\n",
+    "\n",
+    "lammps_calculator.run_lammps(lammps_executable, input_file_name,\n",
+    "                             output_file_name)\n",
+    "\n",
+    "lammps_forces = lammps_calculator.lammps_parser(dump_file_name)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}