How to use mutli-gpu for training?

[tornadomeet]

It seems that hplflownet now only support single gpu training, so how to modify it for using multi-gpu training?

thanks.

[laoreja]

Hi, the bottleneck is that our inputs has variable length since we use sparse CNN. So if we are going to support multi-gpu training, we need to padding correctly I guess, or try to use distributed data parallel so that each GPU use a different process and set the batch size on each gpu still to be 1.

[tornadomeet]

@laoreja thanks, i think i can try the second method, which using torch.nn.parallel.DistributedDataParallel