Integrate StableDiffusion Dreambooth with ColossalAI

[0xdevalias]

WHAT

Integrate with ColossalAI for memory/speed efficiency and improvements.

WHERE

• https://github.com/hpcaitech/ColossalAI

  • Colossal-AI: A Unified Deep Learning System for Big Model Era

• https://medium.com/@yangyou_berkeley/diffusion-pretraining-and-hardware-fine-tuning-can-be-almost-7x-cheaper-85e970fe207b

HOW

• https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion

SEE ALSO

• Community Integration: Making AIGC cheaper, faster, and more efficient. huggingface/diffusers#1212
• [Feature Request]: colossalai integration AUTOMATIC1111/stable-diffusion-webui#4606
• Integrate with ColossalAI for memory/speed efficiency and improvements d8ahazard/sd_dreambooth_extension#174
• [FEATURE]: Integrate StableDiffusion Dreambooth with ColossalAI hpcaitech/ColossalAI#1935
• Integrate StableDiffusion Dreambooth with ColossalAI TheLastBen/fast-stable-diffusion#493

[arpowers]

@0xdevalias i tried colossalAi and it was still pretty buggy in terms of documentation and getting it working as described

[0xdevalias]

That sucks :(

It looks like the dev team of ColossalAI is looking to work directly with huggingface diffusers to simplify integrations in this issue:

• Community Integration: Making AIGC cheaper, faster, and more efficient. huggingface/diffusers#1212

This is very cool @binmakeswell ! Would be very happy to integrate diffusers in colossalAI examples!

Originally posted by @patil-suraj in huggingface#1212 (comment)

Thanks for your enthusiastic response and we look forward to working together!

How do you expect it to be carried out?
Take our collaboration with Lightning for example, we communicated in slack for details, provided documentation and eventually did a joint release.

Originally posted by @binmakeswell in huggingface#1212 (comment)

And there seems to be at least some initial/cursory interest on the issue I opened on the ColossalAI repo RE: integrating with dreambooth:

• [FEATURE]: Integrate StableDiffusion Dreambooth with ColossalAI hpcaitech/ColossalAI#1935

Hi, thanks for reaching out! I think it’s pretty awesome. Can I check it with our team and get back to you ASAP?

Originally posted by @Sze-qq in hpcaitech/ColossalAI#1935 (comment)

So hopefully things rapidly move towards the 'easier to integrate'/'better documentation' direction of things!