Has anyone run this with AMD ROCm (specifically RDNA3 / gfx1100)

[lhl]

I submitted an issue to the ROCm team here ROCm/hipBLASLt#831 but just curious if anyone else (authors or uses) has ever tried running torchtune with ROCm?

I am using a very standard install (latest Ubuntu 22.04.4 LTS w/ HWE kernel, ROCm 6.1.2) and it looks like torchtune doesn't run OOTB since the hiblaslt-dev package doesn't come with gfx1100 kernels. I was able to compile a custom version (and run hipblaslt-test and hipblaslt-bench so I know the lib is working) but get an error like:

rocblaslt error: Could not initialize Tensile host:
std::bad_cast
Segmentation fault (core dumped)

[lhl]

So just to answer my own question. The issue is that even after you compile an updated hipBLASLt, PyTorch (I use the 6.1 nightly) has its own libhipblaslt.so that overrides/causes problems. If you compile torchtune from source (which you need atm for the latest features like sample packing) it will also potentially have it's own copy of hipblaslt that you need to get rid of/replace. I had to attach gdb when running tune to track exactly what was going on.

Also just a quick note, when I was compiling hipBLASLt it was run forever/hang/crash compiling with the default install command - I had to specify only -a gfx1100 to get it to build. On my machine it took about 12 minutes to build). Just leaving this notes in case anyone else decides to try this out.

Also, I found that torch compile appears to give RDNA3 a bigger boost than on the Nvidia side, giving just under a 20% perf boost when compile=True.

[pbontrager]

torchtune assumes you have pytorch already installed, so I don't believe installing torchtune would affect hipBLASLt. Is there an issue with the pytorch libhipblaslt.so, or is it just behind the ROCm version?

[lhl]

Yeah, the problem with PyTorch's hipBLASLt (which I believe is only an issue w/ the ROCm 6.1 nightlies, not stable since it wasn't included?) is that it has a version of hipBLASLt w/o RDNA3 support.