Merge devel to master #1161
Merged
Merge devel to master #1161
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…successfully (deepmodeling#918) Signed-off-by: Chenqqian Zhang <100290172+Chengqian-Zhang@users.noreply.github.com> Now there is a problem, if someone wants to use zbl potential, the energy table will not be copied into the task folder with input.json, I added a new parameter "srtab_file_path" , the user needs to write the path of the energy table, when generating the training task, this file will be copied into the task folder. Signed-off-by: Chenqqian Zhang <100290172+Chengqian-Zhang@users.noreply.github.com>
…eepmodeling#1075) The latest ABACUS 3.0.5 have removed the key word ·ntype·
Fix deepmodeling#1065. Fix deepmodeling#1067. Needs someone to finish the doc...
The original implementation does not work. For example, see deepmodeling#1080.
Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
…deling#1079) Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Fix deepmodeling#1084. Both keys are valid, so we detect which one is used in `default_training_param`.
…eepmodeling#1095) `distutils` will be removed from Python 3.12. See [PEP 632](https://peps.python.org/pep-0632/). Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
The type of parameter "structures" in `relaxation.json` of `dpgen autotest` should be a list. Fix the wrong type in Autotest Overview.
Support "user_forward_files" for `simplify` in the `02.fp` step. Use abspath to create symlinks to avoid unexpected problems.
Remove the old dispatcher and all related tests and examples. Fix deepmodeling#1002. Close deepmodeling#522. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
**expected behavior**:
when `"labeled":true` in `dpgen simplify`, 02.fp will soft-link "labeled
data", and the soft-linked "task dir" will also be created, for format
consistency.
it is expected to be `data.000` and `task.000.000000`,
being respectively guaranteed by funcs `data_system_fmt` and
`fp_task_fmt`
**bug**:
the typo_bug used `data_system_fmt` for the "task dir" instead of
`fp_task_fmt`,
then gives `task.000` instead of `task.000.000000`,
which makes `_check_empty_iter` (who checks `glob.glog("task.000.*")`)
in `generator/run.py` sentence this iter empty,
then `00.train` of the next iter is always skipped.
**consequence**:
this make the "simplify_labeled" process never starts correctly,
no iter0 model presents and randomly-picked data in iter0 are never
trained,
then iter1 gives error that can't finding the graph file from iter0 when
trying copying them due to the train-skip.
**BTW**
thought "simplify_labeled" valuable in some complex or big-data
scenarios but seems not loved by users yet.
pity : (
Signed-off-by: Wanrun Jiang
<58099845+Vibsteamer@users.noreply.github.com>
Signed-off-by: Wanrun Jiang <58099845+Vibsteamer@users.noreply.github.com>
to fix the break of dpgen simplify with "labeled true", when the last iter picked all candidate frames but still have non-zero failed frames output files of the last iter： ``` ls iter.000002/01.model_devi/data.rest/* ... iter.000002/01.model_devi/data.rest/my_elements: box.raw coord.raw energy.raw force.raw type_map.raw type.raw virial.raw ... ``` no `set.000` within, then model_devi of iter.000003 breaks, due to no labeld systems can be loaded, where `iter.000003/01.model_devi/data.rest.old` is the symlink of `iter.000002/01.model_devi/data.rest` https://github.com/deepmodeling/dpgen/blob/355f8eda0c212fe3a072f0865d1ac0d0d7c753b1/dpgen/simplify/simplify.py#L171 **tracking clues** rest_idx only contains not-picked candidates_idx of this iter https://github.com/deepmodeling/dpgen/blob/355f8eda0c212fe3a072f0865d1ac0d0d7c753b1/dpgen/simplify/simplify.py#L292 ``` /dpgen/simplify/simplify.py def post_model_devi(iter_index, jdata, mdata): ... counter = {"candidate": sys_candinate.get_nframes(), "accurate": sys_accurate.get_nframes(), "failed": sys_failed.get_nframes()} <<------- .... # candinate: pick up randomly iter_pick_number = jdata['iter_pick_number'] idx = np.arange(counter['candidate']) <<---- assert(len(idx) == len(labels)) np.random.shuffle(idx) pick_idx = idx[:iter_pick_number] rest_idx = idx[iter_pick_number:] <<-- ``` but "rest_systems" for next iter should contain both "not-picked candidates" and "sys_failed" of this iter: https://github.com/deepmodeling/dpgen/blob/355f8eda0c212fe3a072f0865d1ac0d0d7c753b1/dpgen/simplify/simplify.py#L314 ``` /dpgen/simplify/simplify.py def post_model_devi(iter_index, jdata, mdata): ... for j in rest_idx: <<------ sys_name, sys_id = labels[j] rest_systems.append(sys_candinate[sys_name][sys_id]) <<---- rest_systems += sys_failed <<-- ``` thus, the size passed to `set_size` should be `rest_systems.get_nframes()`, didn't find the necessity of the size_check of the deleted "if line", thought it only the insurance in case of set_zise =0 when passing the size of rest_idx **when breaked, err would be like**: std output of dpgen: ``` INFO:dpgen:-------------------------iter.000003 task 05-------------------------- Traceback (most recent call last): File "/opt/anaconda3/bin/dpgen", line 8, in <module> sys.exit(main()) File "/opt/anaconda3/lib/python3.8/site-packages/dpgen/main.py", line 185, in main args.func(args) File "/opt/anaconda3/lib/python3.8/site-packages/dpgen/simplify/simplify.py", line 535, in gen_simplify run_iter(args.PARAM, args.MACHINE) File "/opt/anaconda3/lib/python3.8/site-packages/dpgen/simplify/simplify.py", line 508, in run_iter post_model_devi(ii, jdata, mdata) File "/opt/anaconda3/lib/python3.8/site-packages/dpgen/simplify/simplify.py", line 250, in post_model_devi sys_entire = dpdata.MultiSystems(type_map = type_map).from_deepmd_npy(os.path.join(work_path, rest_data_name + ".old"), labeled=labeled) File "/opt/anaconda3/lib/python3.8/site-packages/dpdata/system.py", line 1465, in from_format return self.from_fmt_obj(ff(), file_name, **kwargs) File "/opt/anaconda3/lib/python3.8/site-packages/dpdata/system.py", line 1188, in from_fmt_obj system = LabeledSystem().from_fmt_obj(fmtobj, dd, **kwargs) File "/opt/anaconda3/lib/python3.8/site-packages/dpdata/system.py", line 1078, in from_fmt_obj data = fmtobj.from_labeled_system(file_name, **kwargs) File "/opt/anaconda3/lib/python3.8/site-packages/dpdata/plugins/deepmd.py", line 60, in from_labeled_system return dpdata.deepmd.comp.to_system_data(file_name, type_map=type_map, labels=True) File "/opt/anaconda3/lib/python3.8/site-packages/dpdata/deepmd/comp.py", line 50, in to_system_data data['cells'] = np.concatenate(all_cells, axis = 0) File "<__array_function__ internals>", line 5, in concatenate ValueError: need at least one array to concatenate ``` iter.000003/01.model_devi/model_devi.log: ``` WARNING:tensorflow:From /opt/deepmd-kit-2.1.5/lib/python3.10/site-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. Instructions for updating: non-resource variables are not supported in the long term WARNING:root:To get the best performance, it is recommended to adjust the number of threads by setting the environment variables OMP_NUM_THREADS, TF_INTRA_OP_PARALLELISM_THREADS, and TF_INTER_OP_PARALLELISM_THREADS. WARNING:root:Environment variable KMP_BLOCKTIME is empty. Use the default value 0 WARNING:root:Environment variable KMP_AFFINITY is empty. Use the default value granularity=fine,verbose,compact,1,0 /opt/deepmd-kit-2.1.5/lib/python3.10/importlib/__init__.py:169: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged. _bootstrap._exec(spec, module) Traceback (most recent call last): File "/opt/deepmd-kit-2.1.5/bin/dp", line 10, in <module> sys.exit(main()) File "/opt/deepmd-kit-2.1.5/lib/python3.10/site-packages/deepmd/entrypoints/main.py", line 576, in main make_model_devi(**dict_args) File "/opt/deepmd-kit-2.1.5/lib/python3.10/site-packages/deepmd/infer/model_devi.py", line 199, in make_model_devi dp_data = DeepmdData(system, set_prefix, shuffle_test=False, type_map=tmap) File "/opt/deepmd-kit-2.1.5/lib/python3.10/site-packages/deepmd/utils/data.py", line 51, in __init__ self.mixed_type = self._check_mode(self.dirs[0]) # mixed_type format only has one set IndexError: list index out of range ``` Signed-off-by: Wanrun Jiang <58099845+Vibsteamer@users.noreply.github.com> Signed-off-by: Wanrun Jiang <58099845+Vibsteamer@users.noreply.github.com>
Fix deepmodeling#617. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
- add pre-commit config, including black and common fixes. - run pre-commit. - manually fix a YAML file. - skip check json and broken symlinks.
Fix deepmodeling#1108. Note: tests do not cover this case.
…epmodeling#1127) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
per deepmodeling/community#18 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
When turning on the merge_traj function, if the procedure is interrupted in the run_model_devi stage, the old trajectory file will be overwritten after it is rerun. The incomplete trajectory file will cause the following error in the next post_model_devi stage: `.../site-packages/dpdata/lammps/dump.py:176: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. system['coords'] = np.array(system['coords']) DeepModeling ` This PR repaired this question. --------- Signed-off-by: kiwi <46419582+wankiwi@users.noreply.github.com> Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Co-authored-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
deepmodeling#1130) If the user writes a wrong path in "sys_configs", this warning can help locate the problem. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Just support `repeat=True` in deepmodeling/dpgui#119...
fix numpydoc styles
Added the `model_devi_merge_traj` parameter to allow the merging of trajectories into one file using the lmp_input template. --------- Signed-off-by: Pan Xiang <panxiang126@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This is an automatic tool to format and sort imports. See https://github.com/PyCQA/isort for details. --------- Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…t_fp_abacus_scf() (deepmodeling#1153) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
previous natoms only count #atom of the first element, gives wrong results or -inf in multiple-element cases Signed-off-by: Wanrun Jiang <58099845+Vibsteamer@users.noreply.github.com>
njzjz
approved these changes
Mar 20, 2023
wanghan-iapcm
approved these changes
Mar 21, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.