Simulator: pausing/resuming simulation runs

[npinto]

What do you think about allowing pausing/resuming in the simulator (w.r.t. eg blocks) ?

That is:

1. having a way to dump the full state of the simulator at the end of a given number of blocks, and,
2. taking this state and resuming the simulation for another given number of blocks.

This could allow "in-flight" fine tuning of parameters and also much better optimizations of simulations (e.g. "early stopping" of non promising simulations).

[npinto]

/cc @fernand

[fedekunze]

I'll tackle this @npinto @fernand

[rigelrozanski]

woot! please do!

[fedekunze]

actually, this was resolved mostly by #4731, specifically point 1.. What's missing though is the option to declare the initial block from which the sims are resumed. That has been addressed on #4791