"RuntimeError: CUDA error: out of memory" when train TOM with RTX2080Ti

[kris-yangjs]

Hello, thank you for your amazing work! I tried to train the model, and I finished the GMM step. But when I tried to train the TOM with one GPU(2080Ti), I got a error: "RuntimeError: CUDA error: out of memory " , even if I changed the batch size to 1.
And what I want to ask is : what kind of GPU you used to train ?

[pearzzz]

same question

[ovipaul]

V100 GPU(32G)