Add auto restart

[ElliottKasoar]

Resolves #169

Adds auto restart option for MD, which generates restart_stem based on inputs (including file_prefix and restart_stem), and if restarting simulation, uses the most recent matching file for the input structure.

Also includes fixes for:

• prefix_override, which threw errors if given a Paths (e.g. anything via --restart-stem)
• How restart_stem was used, as currently for restart_stem=stem, files of the form stem-T300.0-res-1.extxyz are created, the same as the old usage of struct_name. With these changes, they should now be stem-1.extxyz

Note: the limit after restarting is the effectively the same as before - however far the previous simulation got + the full steps input (e.g. for NaCl-nvt-T300.0-res-100.extxyz, with --steps 1000, the last step would be 1100.)

One thing to consider is that this implementation can (and arguably is likely) to lead to duplicated steps for statistics and/or trajectory file.

E.g. if you were to print statistics every 100 steps, but restart every 1000 steps, then the statistics file may to be written for a while after the last restart file. When we restart, we then append duplicate steps.

Do we want to delete these duplicate steps? This is even harder for the trajectory, since we'd have to calculate which images would be duplicated.

[alinelena]

I would not delete the steps ideally one write restart at the same step as printing, I will add maybe a note to that effect is the two are different. let us not defend the users from themselves

[ElliottKasoar]

Hopefully there would be a corresponding step, but realistically it's quite wasteful to generate a restart file as often as you'd want to output the statistics, so I'd expect this would happen for most use cases unavoidably.

(Unavoidable, and non-trivially fixable being the problem, since the users' choices are reasonable)

[ElliottKasoar]

Just to make sure we're on the same page, I'm talking about something like

janus md --ensemble nvt --struct tests/data/NaCl.cif --steps 1000 --stats-every 10 --restart-every 100
janus md --ensemble nvt --struct tests/data/NaCl.cif --steps 1000 --stats-every 10 --restart-every 100  --restart

If the first command fails at step 140, then the -stats file will have steps 100, 110, 120, 130 and 140.

When we (auto) restart, it resumes from step 100, so we end up with stats for 110, 120, 130 and 140 again.

My issues is (1) I don't think the user should have to set restart_every to match stats_every, and even then this could be inconsistent with traj_every, and (2) there's not a nice way to avoid this otherwise, that I can think of...

Rather than deleting written data, for stats we could maybe check that the corresponding step doesn't already exist before writing again? If we added a label for traj we could potentially do the same for that?

[ElliottKasoar]

From discussions with @oerc0122, perhaps for now it might be sufficient to add a label to each trajectory image (dyn_step?), which means the user can relatively straightforwardly determine from both the stats and traj files where any overlap is?

Either way, it needs to be clearly documented.

[alinelena]

I will call it time_step or timestep otherwise is perfectly ok... maybe since you are at this add another time= time in fs...

[ElliottKasoar]

I will call it time_step or timestep otherwise is perfectly ok... maybe since you are at this add another time= time in fs...

It turned out we already had an info["step"] from when we set the statistics, but that meant it would only update at the stats frequency, so wasn't necessarily accurate at the time each trajectory step is written out.

I think the varying frequencies of updating different info keys is still a problem for others e.g. "density" and "real_time", but we can think about that in a separate issue.

[oerc0122]

Just a couple of minor things.