Tracking issue for targeting AMDGPU devices

[DiamondLovesYou]

This issue tracks progress/roadmap for what needs to be done to codegen for targets like AMDGPUs. Personally, I am working on AMDGPU codegen as it would be used for HSA. Specifically, I am aiming for the amdgcn-amd-amdhsa-amdgiz LLVM target. Note that I’m still learning, so this issue will likely change as guided by experience.

Here are the pieces that will be needed to make this work to a MVP level (ie not providing access to most GPU specific stuff):

• Initialize the LLVM target machine Initialize LLVM's AMDGPU target machine, if available. #51548
• Teach the LLVM codegen backend to be mindful of target machine imposed address spaces PR: Make librustc_codegen_llvm aware of LLVM address spaces. #51576. E.g. allocas are in address space 5 for the target triple I mentioned above.
• Add the amdgpu-kernel ABI (PR Add the amdgpu-kernel ABI. #52032).
• Add a mechanism to delegate virtual function calls (meaning call by pointer value) to runtime libraries.
• Required metadata ??

The address space changes are pretty general. However, in order to not require sweeping changes to how Rust is codegen-ed for LLVM, any target must support a flat address space. Flat meaning an addr space which is a superset of all others.

amdgpu-kernel requires its return type be void. There are two ways I see to do this:

• compile-time checks (somewhere in rustc), ie disallow any return type except ! and ().
• rewriting returns to use an sret-like style: promote the return to be an indirect first argument of the function.

As I recall, Rust inserts wrapper functions for functions with extern “abi” which call the real rust abi function. My current impl went with the magical rewriting, but I think forcing the user to acknowledge this with an error is better long term.

Privately, I've made it to errors stemming from # 4 on general Rust code (ie std/core code). See this repo/crate. Regarding virtual function calls, in principle, it’s possible to support, if using HSA, completely GPU side. amdgpu-kernels have access to two different hsa_queue_ts (one for the host and the device), setup by the GPU’s hardware command processor. When a virtual call is encountered, the trick is to have the GPU write to its own hsa_queue_t then wait on the completion signal. Foreign functions can also be supported in this way, by writing to the host hsa_queue_t instead.

Post-MVP

TBD(TODO) Discuss?

Informational links

• User Guilde for AMDGPU Backend
• AMD’s HIP Clang branch

[LifeIsStrange]

As the work on NVPTX overlap, there could be some common/shared effort.
#38789

[DiamondLovesYou]

I saw that thread! 😄 I have not taken time to read through it, yet, however. Will get to it Soon(TM).

I have an example of my progress, using a Rust fork, from before the i/u128 stabilization ecosystem breakage. This example is from this "trivial" program, and in addition to the AMDGPU specific IR, uses a single source Thingy(TM) (I'm in the process of preparing PRs) to compile functions for a different target than the host.

mir-hsa-ndarray-example.ll. Warning, it's pretty big!

[LifeIsStrange]

As a side note I guess this project could get some development help from AMD or at least some funding from AMD or the AMD subreddit community because rust would be a net improvement for the gpu computing world !
@johnbridgman
@jedwards-AMD

[mati865]

Add the amdgpu-kernel ABI (PR #52032).

This one is merged.

[eddyb]

As I recall, Rust inserts wrapper functions for functions with extern “abi” which call the real rust abi function

FWIW this was fixed years ago, nowadays we have the tools to talk about call ABIs.

[Titaniumtown]

Is this ever going to happen? seems the issue has been inactive for some time now.

[DiamondLovesYou]

In upstream Rust, not likely: I have absolutely no interest in upstreaming anything anymore. I'm (relatively) actively developing my Rust fork which supports AMDGPU however: https://github.com/geobacter-rs.

[Titaniumtown]

Very interesting! I'm cloning and compiling that right now to check it out. Thanks for the info!