[Feature]: add a way to manage thread pools

[Picus303]

Suggestion Description

Context: As training, I'm trying to implement raytracing with HIP by myself. Because of that, I have to start threads hundreds of times per second and the overhead of starting threads becomes a big problem.
I couldn't find an easy way to keep my threads alive and use them to execute the same code over and over. Also, because the GPU threads have no way to passively wait for a signal from the CPU, I had to use active wait which I really wish I could avoid.

Feature: I suggest to either add a way to send events from the CPU to the GPU or add a way to manage pools of threads that aren't just killed when they finish executing their code but are kept alive, ready to run again (same ID, same data).

Note that it's not impossible to do at the moment, just extremely suboptimal. I think it would be very benificial in each case where we have to start threads at a high frequency.

Operating System

No response

GPU

No response

ROCm Component

No response

[cjatin]

Threadpool on GPU might not be the best fit (design wise).

I recommend you to launch work when needed, preferably on a stream. You can also create a graph and get more efficiency of out of it.

[Picus303]

@cjatin Thanks for your advices. I'll try.

Threadpool on GPU might not be the best fit (design wise).

I'm not knowledgeable enough to understand why. Could you please elaborate?

[darren-amd]

Hi @Picus303,

Thanks for the question! From my understanding, using a stream approach provides a more efficient and flexible model for resource management and execution of high-frequency workloads like ray tracing. Also, using graphs could allow for more efficient task scheduling as well as managing task dependencies.

Here is the documentation for Graph Management and Stream Management for further information.