make checkpoint summary information more easily accessible

[jchiang87]

Currently, the only reliable way of checking the progress of a sensor-visit process is by reading in the checkpoint pickle file into python and getting the size of the list of drawn objects. The code could instead provide that summary info either via an easily accessible checkpoint file header (which would require a different format for the checkpoint files), or by writing that info elsewhere, e.g., to the logger output or some other file.

In addition to number of objects drawn, elapsed wall time and/or cputime could be provided.

@tomuram @villarrealas Any preferences or suggestions on how to provide this info or what else should be included?

[villarrealas]

I think the advantage to being in a checkpoint file header is that we won't need to chase around too many things, but changing the storage from pickle to something else feels like it could be potentially troublesome. I think storing it as some sort of checkpoint log that goes in the same directory as checkpoint files (and shares naming conventions) would probably be sufficient.

If we can get any other useful information, I can think of three possible other useful items for diagnostics:

1. Num of objects remaining (to give a rough estimate of remaining runtime, particularly useful with wall time)
2. Num of bright objects drawn
3. Num of bright objects remaining (with some cutoff, if we believe this is what is driving runtime variance).

I figure storing a few more ints during the run probably isn't too awful... though the "bright" objects one does require arbitrary choices in cutoff.

[cwwalter]

Writing down information from CWW/JC in-person conversation.

In order to try to reconstruct e-image files from amp files the random number seeds for for the read and dark noise would need to be recorded. If it is not based on a fixed calculated seed (as in the stand alone converter) those numbers could also be recorded here.

[cwwalter]

This was implemented in PR #150.