State of RNG not taken into account in RFunctions ?

[konkam]

I am unable to get repeatable random number generation using the R random number generators

julia> using StatsFuns

julia> srand(1);

julia> StatsFuns.RFunctions.normrand(1., 1.)
-0.33625734651708594

julia> srand(1);

julia> StatsFuns.RFunctions.normrand(1., 1.)
0.4527588405653422

I am not entirely sure how this is implemented, but I would suspect that the state of the random number generator in Julia is not sent when calling the external function.

This is obtained on Julia 0.5.0 (x86_64-linux-gnu) for:

julia> Pkg.status("StatsFuns")
 - StatsFuns                     0.3.1

[simonbyrne]

Yep, this is a problem since we've split out the Rmath library. I think the best solution here is to modify our Rmath-julia build to allow custom function pointers to the Julia RNG.

[simonbyrne]

Can you try a Pkg.update(), and see if that fixes it?

And if it doesn't, what does Pkg.status("Rmath") give?

[konkam]

Hello,

Pkg.update() does not fix it yet.

Pkg.status("Rmath") gives

julia> Pkg.status("Rmath")
 - Rmath                         0.1.3

[simonbyrne]

Okay, what does:

using Rmath
Rmath.libRmath

give?

[simonbyrne]

And also, what is your versioninfo()?

[konkam]

julia> using Rmath

julia> Rmath.libRmath
"/usr/lib/libRmath-julia.so"

julia> versioninfo()
Julia Version 0.5.0
Commit 3c9d753 (2016-09-19 18:14 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (NO_LAPACK NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Prescott)
  LAPACK: liblapack.so.3
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)

[simonbyrne]

@tkelman Is there any way we can force a rebuild here?

[tkelman]

can delete Pkg.dir("Rmath","deps","usr") and Pkg.dir("Rmath","deps","src") and Pkg.dir("Rmath","deps","downloads") and Pkg.dir("Rmath","deps","deps.jl") then do Pkg.build("Rmath")

[konkam]

thanks, I have a computation running so I'll try as soon as I'm done

[tkelman]

Picking up /usr/lib/libRmath-julia.so is probably from the staticfloat/julia-deps ppa? What Linux distribution are you using, and where did you get Julia from?

[konkam]

I am using Ubuntu 14.04, installed julia from the ppas staticfloat/juliareleases and staticfloat/julia-deps ppa

[tkelman]

You should probably switch to the generic Linux binaries, the ubuntu PPA might not be maintained indefinitely.

[konkam]

Thanks, I'll do just that

[tkelman]

I would think that version of libRmath-julia should be linked against dsfmt and have the patches from Julia though, so I'm not entirely sure what's going on. Switching to the generic Linux binaries may not change this, or you might need to uninstall some of the julia-deps packages to get this package to use a different build of the rmath library.

If there's a way of telling apart a version of libRmath-julia that will work correctly for this package vs one that will not, we can add validation hooks to the bindeps script so it won't use builds that won't work correctly.

[konkam]

If before reinstalling julia from the generic linux binaries I save the /usr/lib/libRmath-julia.so (actually it links to /usr/lib/libRmath-julia.so.2), then compare it to the version once it works, would that help telling them apart ?

Is there a smarter way to proceed before I re-install julia from the binaries ?

[tkelman]

You can modify the path in the Pkg.dir("Rmath","deps","deps.jl") for testing but be aware that will get overwritten any time Pkg.build("Rmath") happens

[richardreeve]

@simonbyrne I believe this is now working and so can be closed.

[simonbyrne]

Yes, this was actually already the case since JuliaStats/Rmath.jl#21 and JuliaStats/Rmath-julia#14, but now much clearer.

[richardreeve]

Yes, I didn't think I had fixed it, but it seemed to be working :)