-
Notifications
You must be signed in to change notification settings - Fork 954
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add random source that matches PyTorch #124
Conversation
This is a great feature for the Swift package @liuliu ! To provide a bit more context for others who might be reading this:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This added random source that matches PyTorch on CPU. In particular, it matches: `torch.randn([], dtype=torch.float)` result. PyTorch's RNG is a bit convoluted and not claimed to be version-stable (will open a separate issue in PyTorch repo on this). However, the current implementation on CPU is fairly straightforward^*. 1. If it is less than 16 elements, it uses Gaussian distribution sampled from MT19937 for double + Box-Muller transformation. 2. If it is more than 16 (16 included), it first do uniform sampling with whatever the resulting data type would be (in this case, torch.float), and then apply Box-Muller transformation over 16-element segment at a type, treating the first floating-point and the 8th as a pair, so on so forth. 3. If it is not a multiple of 16, trace back from the end for 16 elements and redo step 2.
a06279b
to
0960286
Compare
The PR is rebased and updated! |
It's so nice to see the original author Liuliu who made the the DrawThings app for Stable Diffusion in swift to be finally at this repo. Any chance we can ask your help on getting additional samplers such as Euler A implemented? Which would be awesome. |
Thanks for the kind words! It is hard to keep up with the landscape of samplers for sure, since the newest of the day changed again: AUTOMATIC1111/stable-diffusion-webui#7710 Anyway, Euler A is a pretty simple sampler (comparing to DPM++ SDE Karras), here is the gist of the code (it would take more effort to migrate my code over):
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for addressing the feedback @liuliu, we are almost there! I just tested these changes and the CLI argument does not seem to be able to modify the default RNG (.numpyRNG
). Do you mind fixing that? After that, this looks ready to go :)
Good catch! Missed the update! |
This added random source that matches PyTorch on CPU. In particular, it matches:
torch.randn([], dtype=torch.float)
result.PyTorch's RNG is a bit convoluted and not claimed to be version-stable (will open a separate issue in PyTorch repo on this). However, the current implementation on CPU is fairly straightforward^*.
If it is less than 16 elements, it uses Gaussian distribution sampled from MT19937 for double + Box-Muller transformation.
If it is more than 16 (16 included), it first do uniform sampling with whatever the resulting data type would be (in this case, torch.float), and then apply Box-Muller transformation over 16-element segment at a type, treating the first floating-point and the 8th as a pair, so on so forth.
If it is not a multiple of 16, trace back from the end for 16 elements and redo step 2.
#########