Futhark on Intel GPU

[DDEle]

Hi there, I found this exciting project while exploring repos with the CUDA tag. Based on my interest of functional programming and experience of AI performance on Intel GPUs, I would like to make contributions to Futhark as a side project. Starting from basic usages of Futhark and recaption of my undergraduate FP and compiler knowledge, my final goal is to add native support of Intel (integrated / discrete) GPUs to Futhark. Here is my draft plan and I'd like to have your concerns and suggestions.

• Trying to run futhark example programs with OpenCL backend on my personal devices (UHD770 & Arc770) on
  • WSL2
  • Native Windows
• Go through the docs and manuals of Futhark and actually write some futhark programs (probably some fancy deep learning primitives)
• Get Futhark benchmarks run on Intel GPUs
• Maybe try to solve some existing issues to get me more familiar with the Futhark compiler
• Eventually implement a new backend of DPCPP (i.e. Intel's dialect of SYCL) and hopefully merge it to the main steam based on how much it can outperform the OpenCL backend

[athas]

If Intel has a decent implementation of OpenCL, I'd expect it to work already. Some tuning may be necessary to obtain optimal performance - the compiler is currently based around AMD/NVIDIA-style architectural assumptions. Most of the initial development of Futhark took place on mobile Intel GPUs, and they worked quite well.

I find it unlikely that a SYCL backend would be faster than OpenCL. Why would it be?

[DDEle]

Thank you Troels for your suggestions.

If Intel has a decent implementation of OpenCL, I'd expect it to work already.

Yes, I did make it work on my WSL on both integrated/discrete graphics. I'm still trying to figure out the benchmark/profiling system of Futhark.

I find it unlikely that a SYCL backend would be faster than OpenCL. Why would it be?

I'm not sure, and I do need much more investigations before actual development of the new backend.

For now, I can only come came up 2 reasons to support SYCL over OpenCL

1. In terms of games, I heard that Arc A770 can performs better on Vulkan/DX12 than OpenCL based games. So maybe there is a similar performance gap in terms of computation jobs? Specifically, Intel is paying more attentions on SYCL compilers / drivers and having a SYCL backend can utilize these optimization works.
2. SYCL programs/libraries generated by Futhark SYCL backend can be easier to integrate into DPCPP projects.

In addition, I notice that there is a HIP Futhark backend and maybe similar reasons may apply?

[athas]

Yes, I did make it work on my WSL on both integrated/discrete graphics. I'm still trying to figure out the benchmark/profiling system of Futhark.

https://futhark.readthedocs.io/en/latest/man/futhark-profile.html (but perhaps also read https://futhark-book.readthedocs.io/en/latest/practical-matters.html#benchmarking )

In terms of games, I heard that Arc A770 can performs better on Vulkan/DX12 than OpenCL based games.

I assume you mean OpenGL here. I think most OpenGL implementations are stagnating, since nobody really likes the API, and it is clearly superior alternatives (Vulkan and WebGPU). It is certainly possible that Intel's OpenCL implementation is similarly bad (which might happen if they are all-in on SYCL), but my understanding was that these were all based on the same underlying compiler and runtime infrastructure. OpenCL may not have much market penetration, but the kind of runtime system you need to make OpenCL work well is pretty much a subset of what you need for SYCL or anything similar (the OpenCL frontend language is pretty trivial). Also, Intel put pretty serious effort into OpenCL last I checked, so it would be a change if they didn't care at all anymore.

SYCL programs/libraries generated by Futhark SYCL backend can be easier to integrate into DPCPP projects.

It's possible, but since Futhark can by necessity only expose monomorphic and first-order functions to the outside world, you don't get all the template-based SYCL flexibility. If SYCL has any way of interoperating with OpenCL (for transferring data), then a simple wrapper generator in the style of futhark-bindgen would probably be as ergonomic as an actual SYCL backend.

In addition, I notice that there is a HIP Futhark backend and maybe similar reasons may apply?

That is certainly possible. The HIP backend outperforms OpenCL on AMD hardware because HIP (and CUDA) has certain guarantees that are not provided by OpenCL. The main consequence of these is these APIs allow a very efficient scan implementation, which is a critical building block for fancy parallel programming. Some details here: https://futhark-lang.org/blog/2023-08-30-futhark-0.25.3-released.html

[DDEle]

Thanks for your details, which probably will save a lot of time for me. I'll definitely consider your idea of bindgen and check if Intel has something for scan.

Very glad to have a discussion with you here before spending effort on a wrong direction. I'll let you know when I have more progress / thoughts hacking Futhark.