Add context propagation support for direct Thread creation

[iNikem]

We support context propagation when Runnable or Callable is submitted for execution to some executor service. But this does not work with the most primitive way to dispatch a unit of work into a separate thread: new Thread(Runnable).start(). Should add this instrumentation as well.

[trask]

this will make our context leaking problems worse...

[JonasKunz]

I'm wondering whether this issue will gain importance with the arrival of virtual threads?

Prior to virtual threads, it was more of an antipattern to use new Thread(...).start() instead of ExecutorServices: threads are expensive and slow to start.

This changes now with virtual threads. It might become more common do the following:

Thread task = Thread.startVirtualThread(() -> ...)
...
task.join();

At the same time, the ExecutorService approach remains relevant if you are interested in querying the results of the asynchronous task, which already is covered by existing instrumentations:

try(var executor = Executors.newVirtualThreadPerTaskExecutor()) {
   Future<?> result1 = executor.submit(...);
   Future<?> result2 = executor.submit(...);
} //waits for all submitted tasks to terminate

Do you have any opinions here?

[mateuszrzeszutek]

We might actually need to modify the executors instrumentation a bit, since Executors.newVirtualThreadPerTaskExecutor() uses a new ThreadPerTaskExecutor executor service implementation.

For the direct Thread usage: I think we could instrument just the new virtual thread methods, without touching any of the platform thread stuff.

[JonasKunz]

For the direct Thread usage: I think we could instrument just the new virtual thread methods, without touching any of the platform thread stuff.

We've been thinking about the same thing with the elastic APM agent. However, even with virtual threads the point against it about context leakage remains:
E.g. imagine a service where a span is created to capture the startup procedure of this service.
The startup procedure includes spinning up a Thread (virtual or non-virtual) for accepting HTTP-connections. You would definitely not want to have the context from the initial startup propagate to the HTTP request handling I assume.

An option we considered is to provide an instrumentation for Thread.start() based instrumentation, but to make it configurable:

• Perform context propagation for all thread
• Perform context propagation only for virtual Threads
• Do not propagate context

This gives users the choice, but it still is difficult to come up with a reasonable default.

[mateuszrzeszutek]

The startup procedure includes spinning up a Thread (virtual or non-virtual) for accepting HTTP-connections. You would definitely not want to have the context from the initial startup propagate to the HTTP request handling I assume.

Aren't virtual threads supposed to be used for short-lived tasks though? If you start a new virtual thread for the purpose of handling HTTP connections that seems to kinda defeat the purpose (you could use a standard thread pool instead after all).

Anyway, you've a good point though -- perhaps we should just wait a bit and see what usage patterns emerge after a while?

[JonasKunz]

Aren't virtual threads supposed to be used for short-lived tasks though? If you start a new virtual thread for the purpose of handling HTTP connections that seems to kinda defeat the purpose (you could use a standard thread pool instead after all).

To my understanding virtual threads are not limited to short-lived tasks. Short-lived tasks are just the key selling point for virtual threads where platform threads previously couldn't be used.

From a Java user perspective, the only remaining benefit of platform threads is fairness in case of CPU-contention: Your platform thread is guaranteed to run at some point in time due to preemption of platform threads. Virtual threads in contrast can hog the CPU if they never finish and never block, causing other virtual threads to never run. I would however say that this is not your typical usage scenario in an HTTP server.

Maybe a service receiving message from multiple message sources / queues is a better example:
Imagine a service who consumes messages from thousands of sources: On startup, you create a virtual thread for each source. Each thread simply does a blocking read on its message source in a while(true) loop. This would not be possible efficiently with platform threads.

perhaps we should just wait a bit and see what usage patterns emerge after a while

Yeah I'm feeling the same way.