Rayon and random number generator

[rohitjoshi]

What is the best way to pass thread-safe/thread-local random number generator so we can execute distribution operation with multiple threads?

e.g . How can I run below code to generate normal distribution in parallel using rayon or alternate way?

fn normal(mu:f64, sigma: &Vec<f64>) -> Vec<f64> {
        let mut rng = thread_rng();
        let mut v = Vec::with_capacity(sigma.len());
        for i in 0..sigma.len() {
            let mut g = Normal::new(mu, sigma[i] ).unwrap();
            v.push(g.ind_sample(&mut rng));
        }
        v
    }

I tried using below code but due to mutable rng, it is not allowed.

fn normal(mu:f64, sigma: &Vec<f64>) -> Vec<f64> {
         let mut rng = thread_rng();
        let mut v = Vec::with_capacity(sigma.len());
        (0..sigma)
            .into_par_iter()
            .map(|_| {
              let mut g = Normal::new(mu, sigma[i] ).unwrap();
               g.ind_sample(&mut rng)
             })
            .collect_into_vec(&mut v);
          v
}

[pitdicker]

I think the easiest way would be to get your handle to ThreadRng in the loop.
Not tested, but something like this:

fn normal(mu:f64, sigma: &Vec<f64>) -> Vec<f64> {
    let mut v = Vec::with_capacity(sigma.len());
    (0..sigma)
        .into_par_iter()
        .map(|_| {
            let mut rng = thread_rng();
            let mut g = Normal::new(mu, sigma[i] ).unwrap();
            g.ind_sample(&mut rng)
        })
    .collect_into_vec(&mut v);
    v
}

You may also want to move let mut g = Normal::new(mu, sigma[i] ).unwrap() outside the loop, but it will probably not matter much.

---

Actually I think this is an important scenario where we don't have a good solution yet. This works with ThreadRng, but not with other RNGs that would be better suited for the task, like in this case Xoroshiro128+.

What some solutions do is generate a vector of random numbers first, and then use those in Rayon's parallel iteration. But that does not work with the way most of our Distributions work (they may use an unknown number of values from the RNG).

[pitdicker]

I think it would not be hard tot improve Rand for this use. We would have tot add a ParallelIterator variant of sample_iter. I'll try piece something together.

[rohitjoshi]

@pitdicker Thanks. your example did work and was able to reduce the time from 4 to 2 second. let mut rng = thread_rng(); would this be created once per thread or every iteration?

[pitdicker]

Thanks. your example did work and was able to reduce the time from 4 to 2 second.

Nice 😄.

let mut rng = thread_rng(); would this be created once per thread or every iteration?

ThreadRng stores its internal state in thread local memory, and thread_rng() only gets a reference to that. So what this does is set op a ThreadRng for every thread rayon uses, and than for every iteration in every task look up that RNG.

I have been playing some more with rayon yesterday, and now have a commit that adds basic support in pitdicker@a216bfa. It needs some cleaning up, but is an alternative to the thread-local trickery so that it can be used with normal PRNGs.

[dhardy]

Great addition @pitdicker!

I think for now you might just as well do g.ind_sample(&mut thread_rng()) @rohitjoshi.

[dhardy]

#399 is open; I think any further discussion can take place there.