The interval for torch::jit::fuser::cuda::randlike is not the same as for torch.rand_like

[IvanYashchuk]

🐛 Describe the bug

See the discussion here #1799 (comment).

Turns out nvfuser's randlike function generates "1.0" in uniform samples while in PyTorch uniform distribution is sampled on the interval [0.0, 1.0), "0.0" is included, "1.0" is excluded.

I think nvfuser's random numbers should be changed to sample on [0.0, 1.0) interval. uniform and uniformf functions are implemented here, and used for randLike device functions: https://github.com/csarofeen/pytorch/blob/fa4e6a4739a9daaa0e4111fb4730704d79c91010/torch/csrc/jit/codegen/cuda/runtime/helpers.cu#L302-L308
We could replace generated samples of 1.0 with 0.0:

 __device__ float randLikef(Philox& rnd) { 
   float sample = uniformf(rnd());
   return sample == 1.0 ? 0.0 : sample;
 } 

Versions

devel branch

[jjsjann123]

I guess with #1804 this is considered as fixed?

[IvanYashchuk]

Yes, thank you!