Random state is reset even when doing a warm_start

[louisabraham]

If a random state is specified, it will be reset each time fit is called. See here:

https://github.com/EpistasisLab/tpot/blob/master/tpot/base.py#L594-L596

Therefore the individuals will always be the same.

The consequence is that it is not possible to get both reproducibility and periodic save.

[weixuanfu]

The individuals should be different when doing a warm_start since the initial population won't be reset. (see here). TPOT need reset random state each time fit is called for reproducibility.

[louisabraham]

I saw the code for the population, but I experienced a strange behavior.

I used max_time_mins and a loop to restart a fit every hour, and observed absolutely no improvement (I was very far from a good score according to other runs) and killed it after a few hours.
Moreover, one estimator used to run for a very long time (see #780), and it had the same index every time, which made me think that the population didn't change.

[weixuanfu]

Hmm, how many generations in each loop? how long is max_time_mins? I think the population won't change if one generation does not finish.

[louisabraham]

I don't remember the exact number but there were more than 8 generations.
max_time_mins was 60.

I don't know if I can reproduce this easily (private dataset).