Settings for Multiple GPU Ports usage

[Abubakkar13]

Hey,
Thanks for this great library

I want to know how to make use of 2 or more GPUs at the same time in SimpleT5 model training, Since I am a GPU poor😅 so I tried using Kaggle (https://kaggle.com) where we may get two ports of T4 GPU, so as per the code from README from the SimpleT5's GitHub repository we could make use of GPU by use_gpu attribute, but currently using the single port of GPU and the other port is not being used.

So is there any way to set cuda ports to make the model to use 0 and 1 and 2 ... so on?