Reproducibility of experiments

[wizmik12]

Everytime that I run experiments I obtain different metric values. SVGP has a stochasticity component in the selection of mini-batches, where the seed is fixed so the estimated parameters and metrics values like accuracy are always the same. Here, in the doubly stochastic I found this problem to me for doing reproducibility research.

I found how to solve it. In the reparametrization trick, we can fix a seed in the sample of the unit normal (where the propagation through the layers starts). But this is not enough, I also have to set a numpy seed on my script. I do not understand where the numpy seed affects to the doubly stochastic code. Could someone tell me why?

So I would like to know how to modify the code of this package so it would be not necessary to fix numpy seed in every script (because sometimes I forget). I also hope that this could be of help to some people.

[hughsalimbeni]

Setting seeds in tensorflow is always a bit tricky and counterintuitive: https://www.tensorflow.org/api_docs/python/tf/random/set_random_seed. The DGP code wasn't written with seed reproducibility in mind (though I admit that it should have been), and tensorflow may have changed a bit since this code was written. Thanks for your observations though, and I'd welcome a PR. Regarding the numpy seed it might be for k-means or for some other initialization from numpy.