Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of random seeds #77

Closed
Zaharid opened this issue Feb 9, 2018 · 2 comments
Closed

Improve handling of random seeds #77

Zaharid opened this issue Feb 9, 2018 · 2 comments

Comments

@Zaharid
Copy link
Contributor

Zaharid commented Feb 9, 2018

Validphys should try very hard to make results reproducible. This in turn requires that random generators are seeded in a well defined way. Each provider that uses random numbers should have its own random state and not depend on some global state (and e.g. the order of the actions which is undefined in reportengine). However:

  • The seeds must be different for different providers to avoid spurious correlations.
  • Users cannot be required to write a random seed for every function.

A possible solution is to mix the string with something that depends on the provider such as its name, as in:

provider_seed = int.from_bytes(provider.__name__.encode(), byteorder='little')
seed += provider_seed

(where provider is to be retrieved with some metaprogramming magic).

Ideally it should be possible to:

  • Not supply seeds as inputs in the runcard.
  • Have the automatically generated seed written somewhere.
  • Be able to reproduce the result with the generated seed.
@Zaharid
Copy link
Contributor Author

Zaharid commented Feb 10, 2018

The idea above is too simplistic in that we may want to run the same function with different parameters and different random seeds. It gets very complicated when considering that we also may want to run it with the different paamenters and the same random seeds.

@scarlehoff
Copy link
Member

This is now done. We are quite reproducible unless tensorflow / numpy are updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants