Storing/using last_started and last_ended

[jmlarson1]

No description provided.

[jlnav]

Any anecdotal speed-comparisons between this approach and develop?

[shuds13]

mpi4py with 4 workers, wallclock time:

I compared original code with sliding window and with a one-liner that simply returns if canceled is False for all H, and with not calling kill_canceled_sims at all (time if we were to default to False).

time mpirun -np 5 python test_1d_sampling.py

Code	1e4 sims	1e5 sims
Original code	6.330s	1m41.355s
Sliding window	6.286s	1m13.934s
With oneliner	5.830s	1m23.056s
Dont call func	5.808s	1m7.471s

1,000 sims - Sliding window benefits little. 10,000 sims, sliding window benefits significantly!

Local comms. With no sleep added - times show large overheads accumulating. If adding a small sleep, times are:

Code	1e4 sims	1e5 sims
Original code	0m5.874s	1m47.824s
Sliding window	0m6.217s	1m29.937s
With oneliner	0m5.886s	1m28.981s
Dont call func	0m5.798s	1m17.970s

The one-liner benefits more, when no cancels. If we keep kill canceled sims as the default - maybe we should combine one-liner with sliding window, to make quicker when no cancels. If we make kill canceled False, by default, the one-liner wont help.

[shuds13]

I'm going to test a bit more, to see if it's worth combining the one-line check with the sliding window.

changes required