Seeding random number generators in NEST3

[heplesser]

See also #245.

In the past, NEST has used RNGs accepting 32-bit integers as seeds. This is not sufficient: 32 bits allow for only 4.2e9 different seeds. In parallel simulations, one seed is required per VP. In NEST2 the user needs to assign seeds manually to VPs and ensure to use non-overlapping seed ranges for different simulations. This is complicated and error prone.

Another problem is the lack of seeds. Our simulations on K (Kunkel et al, 2014) used close to 7e5 VPs, so with one seed per VP, we have only about 4.2e9/7e5 = 6000 different seed sequences available, which is an unacceptably small.

We therefore need to use RNGs 64 bit seeds, as they are provided by the C++11 STL. We can then proceed as follows:

• The user specifies a single seed, limited to, e.g., U=32 bits. The user can choose this seed at will, thus has 4e9 different seeds available.
• NEST uses the remaining N=64-U bits to create unique seeds for all VPs and the GRNG.
• One could even set aside a small number of bits to allow for seed-series for different purposes, e.g. for providing seeds for Python RNGs.

The following questions arise

1. How to split between N and U?
2. How to combine the N and U bits to form the 64-bit seed?
3. What seeds to make available to the Python level?

Concerning 1, the size of the human brain with O(1e11) neurons poses a definite upper limit for network size. Furthermore, it seems little probable that we will simulate fewer than 100 neurons per VP in NEST, indicating at most 1e9 VPs. This number exceeds the currently largest simulations by a factor 1000 and will likely not be reached for a good number of years. Therefore, the following scheme seems sensible:

• U = 32 bit
• N = 30 bit
• 2 bits reserve

Concerning 2, I would propose the following:

1. The seed provided by the user (32 bit) is never used "naked" in the kernel. The user can therefore use this seed e.g. for the Python RNG, as long as only a single seed is needed at the Python level.
2. All seeds for internal use are obtained by extending the user-defined seed with numbers increasing from 1, where 1 yields the GRNG seed and 2, 3, ... the VP-specific seeds.
3. Two alternatives are possible for seed-extension:
   1. The "seed counter" is simply added to the higher 32 bits of the 64 bit seed. This is straightforward, but might lead to problems if seeding mechanisms do not handle all bits equal (should not be the case with modern RNGs, but ...).
   2. The "seed counter" is interleaved with the user-supplied seed, e.g., by placing the seed counter bits at positions given by 2*k*pi mod 64 to obtain an even but irregular placement.
   3. In both cases, some care needs to be taken to ensure consistent seeds on big- and little-endian machines.

Concerning 3, it is in principle conceivable (though not advisable) that a user would want per-VC seeds also at the Python level. To distinguish these seeds from internally used seeds, one could reserve 2 bits allowing one to create up to four different "seed series".

[peteroupc]

For your information.

I have a definition of high-quality PRNGs in my article on RNG recommendations. I also list examples of high-quality PRNGs. One of the requirements of the definition is that the PRNG should admit at least 2^63 seeds.

As relevant here, I should note that so-called "counter-based PRNGs" (a concept introduced in "Parallel Random Numbers: As Easy as 1, 2, 3", 2011) give each seed its own independent sequence of random numbers (and take a counter that indexes into that sequence). Examples include SFC, Philox, and hash functions that hash a seed and a counter. Many of these functions take seeds of much longer than 64-bits, and hash functions can (at least theoretically) take seeds and counters of arbitrary size.

[jougs]

@peteroupc thanks for the input. We are currently in the process of refactoring the random library of NEST and will then also natively support all C++11 RNGs and distributions as well as Random123.

The idea would be that the user only specifies a master seed. From this we then generate per-thread or per-generator seeds with 64 bits internally. Our main reason for this is to remove complexity for users and make scripts independent of the number of threads. Moreover, this makes it impossible to accidentally specify the same seed for different generators, which would lead to unwanted correlations in the streams.