Multi-GPU usage support?

[neutrinotek]

The server I'm running this on has 2 GPUs, but only seems to be utilizing one of them. Since they are older GPUs (GTX1070), they only have 8gb of vram, so using the GPU enabled version of this model results in an out of memory error. Is there currently any way to implement multi-gpu support in this app?

[bigcat88]

First we need diffusers to support it: huggingface/diffusers#6240

After that we can remove this:

ai_image_generator_bot/lib/main.py

Lines 86 to 89 in 2084081

	if torch.cuda.is_available():
	PIPE.to("cuda")
	elif torch.backends.mps.is_available():
	PIPE.to("mps")

and just do

pipe = pipeline(...., device_map="auto")