Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Implementation of Token Merging for fast SD #61

Closed
1 task done
GreenLandisaLie opened this issue Mar 31, 2023 · 3 comments
Closed
1 task done
Labels
enhancement New feature or request

Comments

@GreenLandisaLie
Copy link

Is there an existing issue for this?

  • I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Assuming its possible - an implementation of https://github.com/dbolya/tomesd would be great!
They claim significant memory savings and inference speed improvements with very little quality loss.
However, it requires pytorch >= 1.12.1 (for scatter_reduce) so I'm not sure this can be ported to DirectML though.

Proposed workflow

.

Additional information

No response

@GreenLandisaLie GreenLandisaLie added the enhancement New feature or request label Mar 31, 2023
@lshqqytiger
Copy link
Owner

lshqqytiger commented Mar 31, 2023

DirectML seems to be weak on handling partially modified dimensions. (dim < 0)
I added some fallback (to cpu) for it. Then, it was slower and generated some broken images.

@GreenLandisaLie
Copy link
Author

Thank you for taking a look into it.
Since its not something solvable on your end, should I close this issue or keep it open in hopes the DirectML team fixes this someday?

@lshqqytiger
Copy link
Owner

I think they'll fix it someday, but I don't know when it will be. I don't think we need to maintain this issue because an extension for Token Merging is already out.

lshqqytiger pushed a commit that referenced this issue Apr 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants