Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix autocast v fp16 + add benchmark config instructions in benchmark.md #11

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

eolecvk
Copy link
Collaborator

@eolecvk eolecvk commented Oct 12, 2022

  • Fixed conditional logic when using autocast in benchmark code.
  • Updated diffusers==0.3.0 to diffusers==0.4.2
  • @chuanli11 : Any reason to not include torch and torchvision in the requirements?

Removing autocast does seem to reduce the latency and memory in a big way!
Here is what the new benchmark results for autocast v fp16 look like.

device precision autocast runtime n_samples latency memory
NVIDIA RTX A6000 single False pytorch 1 7.37 7.73
NVIDIA RTX A6000 half True pytorch 1 4.39 4.52
NVIDIA RTX A6000 half False pytorch 1 3.34 3.38
NVIDIA RTX A6000 single False pytorch 2 14.0 9.57
NVIDIA RTX A6000 half True pytorch 2 8.29 8.74
NVIDIA RTX A6000 half False pytorch 2 6.17 4.92
NVIDIA RTX A6000 single False pytorch 4 27.25 18.4
NVIDIA RTX A6000 half True pytorch 4 15.73 11.4
NVIDIA RTX A6000 half False pytorch 4 11.59 8.82

Also added benchmark config instructions in benchmark.md

@chuanli11
Copy link
Contributor

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

@eolecvk
Copy link
Collaborator Author

eolecvk commented Oct 12, 2022

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

It's because the original repo was setup that way:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/setup.py

...but even in their case, they install pytorch and torchvision in conda's environment.yaml:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/environment.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants