Is it possible to queue requests using gradio? (RuntimeError when using concurrency_count)

[aashishvasu]

I'm trying to speed up my workflow while using img2img_gradio.py and txt2img_gradio.py and was hoping to be able to queue requests so that I can leave things unattended.

According to the gradio examples, to enable queuing, the launch function needs to be modified to

demo.queue(concurrency_count=5).launch()

However, when I modify that code, I get this error

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

I'm not sure what's going wrong, because otherwise running things manually, cuda seems to be set up properly. If anyone could point me in the right direction that would be amazing!