[wasm-mt] Allow JSImport promises to run on managed threadpool worker threads

[lambdageek]

Depends on #83998 - we need Emscripten 3.1.33 or later (specifically this commit: emscripten-core/emscripten@0c2f589) in order to work correctly. Otherwise thread keepalive is not tracked correctly and pthreads will exit before they're supposed to.

This PR allows managed thread pool worker threads to call out to asynchronous JS APIs.

Contributes to #77287, #68162, #76956

This PR implements the following:

• Refactors the PortableThreadPool.WorkerThread.cs file into
  • PortableThreadPool.WorkerThread.NonBrowser.cs that contains PortableThreadPool.WorkerThread.CreateWorkerThread() and the thread body function WorkerThreadStart() for normal non-JS platforms that repeatedly waits on a LowLevelLifoSemaphore that and either begins dispatching work by calling (WorkerDoWork) or trying to stop the worker thread by calling WorkerTimedOutMaybeStop if the semaphore is not released (ie the worker has been idle for some time) before a timeout is reached.
  • PortableThreadPool.WorkerThread.cs - common code from the threadpool worker thread's loop. The main loop is broken up into two pieces: WorkerDoWork - repeatedly calls TakeActiveRequest() and ThreadPoolWorkQueue.Dispatch() to process queued work items; and WorkerTimedOutMaybeStop which is called when the worker has been idle and no new work has been signaled and which tries to stop the worker or else go around the main loop one more time if new work shows up (or there's unfinished async IO).
• Creates a new PortableThreadPool.WorkerThread.Browser.Threads.Mono.cs file that implements the CreateWorkerThread and WorkerThreadStart functions for a JS-based async event loop. Instead of looping in a managed while loop, the thread instead returns to the JS event loop after installing callbacks that are called when the threadpool semaphore is released or after a timeout. In the meantime the JS event queue can run code on the worker in order to settle promises.
  • The main difference from non-Browser is that we use a new version of LowLevelLifoSemaphore that uses asynchronous callbacks for success and timeout.
  • Additionally the thread is started using WebWorkerEventLoop.StartExitable and uses a WebWorkerEventLoop.KeepaliveToken to return from the thread start function to the JS event loop while keeping the POSIX thread alive. This, together with the callback-based semaphore allows the thread to stay alive while JS events are resolved. When the semaphore is released, the wait callback is called and we process managed threadpool work.
  • If the semaphore times out, the timeout callback is called and we decide if there is any pending async IO (in the form of unsettled JS interop promises) and either schedule another semaphore wait or destroy the keepalive token allowing the thread to exit if the keepalive count reaches zero.
• Refactors LowLevelLifoSemaphore into two versions:
  • the "normal" version (created with the public constructor) that can be used with a synchronous Wait method,
  • and the "async wait" version (created with LowLevelLifoSemaphore.CreateAsyncWait()) that cannot use Wait but can use the callback-based PrepareAsyncWait method.
  • The common code for Wait and PrepareAsyncWait for managing the semaphore's counts is shared.
• Implements the underlying native LifoSemaphoreAsyncWait struct that shares some common code with the "normal" LifoSemaphore (via LifoSemaphoreBase).
  • To implement mono_lifo_semaphore_asyncwait_prepare_wait and mono_lifo_semaphore_asyncwait_release we use two emscripten APIs: emscripten_set_timeout to schedule a C callback to run on the current thread after a certain amount of time has passed; emscripten_dispatch_to_thread_async to allow one thread to queue a C callback to run on another pthread (note this depends on the main browser thread being responsive enough to relay JS messages from one thread to another).
  • When a thread starts to wait we create a wait queue entry and simultaneously start a timeout and return from prepare_wait.
  • To release the semaphore, we select a wait entry and dispatch its success callback to run on the waiting thread. When the success or timeout callbacks run, they unlink the wait entry (and the success callback cancels the timeout callback) and call the user-provided success/timeout functions.
  • (There is a case where the async success is queued but the timeout fires first that is handled by the timeout callback first checking if a success has been queued to run, too, in which case the timeout silently becomes a no-op)
• The managed threadpool worker keeps its pthread alive as long as the _js_owned_object_table is gc-handles.ts is non-empty: that is if there are any JS promises that will eventually callback to a C# Task on the current worker, even if the worker doesn't have any active threadpool work, it will keep the pthread alive until all the JS promises settle (and remove themselves from _js_owned_object_table.

[lambdageek]

I pushed a hack that kind of works around the fact that emscripten_runtime_keepalive_push/pop are no-ops. But it's super fragile. Mostly it's just here to let me keep working on this branch while we take another Emscripten bump

[lambdageek]

This is getting closer to being usable.

Remaining required things:

• make sure we keep track of unsettled JS promises and set IsIOPending appropriately - disallow a worker to exit if it is waiting for a slow JS promise
• verify that if we flood a worker with work it will still periodically yield to JS. OR else we need to rewrite WorkerDoWork (while (TakeActiveRequest(...))) to periodically yield

Remaining nice to haves:

• Share more code with the normal WorkerThread code - share the loopy logic
• make the LowLevelJSSemphore use the managed spinlock logic and maybe make it a special variant of the normal LowLevelLifoSemaphore managed class (with a different native impl, and an additional JS-friendly wait method)
• Figure out if there's some way to make Task.Wait() throw an InvalidOperationException if the thing at the head of the chain is a JS Promise from the same thread.

[lambdageek]

Verified that flooding a threadpool worker with work doesn't prevent it from servicing the JS event loop.

Approach:

1. ThreadPool.SetMaxThreads(2,2)
2. Create a FloodWork task that uses Task.Run to start two more copies of itself and then does an expensive synchronous computation before awaiting its children.
3. Start a task that calls out to globalThis.fetch() and then calls and response.text() after a (JS setTimeout-based) delay of a few seconds.
4. From the main thread, race FloodWork and the background fetch
5. cancel the flood
6. observe that we got the expected string back from response.text()

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Depends on #83998 - need Emscripten 3.1.33 or later (specifically this commit: emscripten-core/emscripten@0c2f589) in order to work correctly. Otherwise thread keepalive is not tracked correctly and pthreads will exit before they're supposed to.

This PR allows managed thread pool worker threads to call out to asynchronous JS APIs.

This PR implements the following:

• Refactors the PortableThreadPool.WorkerThread.cs file into
  • PortableThreadPool.WorkerThread.NonBrowser.cs that contains PortableThreadPool.WorkerThread.CreateWorkerThread() and the thread body function WorkerThreadStart() for normal non-JS platforms that repeatedly waits on a LowLevelLifoSemaphore that and either begins dispatching work by calling (WorkerDoWork) or trying to stop the worker thread by calling WorkerTimedOutMaybeStop if the semaphore is not released (ie the worker has been idle for some time) before a timeout is reached.
  • PortableThreadPool.WorkerThread.cs - common code from the threadpool worker thread's loop. The main loop is broken up into two pieces: WorkerDoWork - repeatedly calls TakeActiveRequest() and ThreadPoolWorkQueue.Dispatch() to process queued work items; and WorkerTimedOutMaybeStop which is called when the worker has been idle and no new work has been signaled and which tries to stop the worker or else go around the main loop one more time if new work shows up (or there's unfinished async IO).
• Creates a new PortableThreadPool.WorkerThread.Browser.Threads.Mono.cs file that implements the CreateWorkerThread and WorkerThreadStart functions for a JS-based async event loop. Instead of looping in a managed while loop, the thread instead returns to the JS event loop after installing callbacks that are called when the threadpool semaphore is released or after a timeout. In the meantime the JS event queue can run code on the worker in order to settle promises.
  • The main difference from non-Browser is that we use a new version of LowLevelLifoSemaphore that uses asynchronous callbacks for success and timeout.
  • Additionally the thread is started using WebWorkerEventLoop.StartExitable and uses a WebWorkerEventLoop.KeepaliveToken to return from the thread start function to the JS event loop while keeping the POSIX thread alive. This, together with the callback-based semaphore allows the thread to stay alive while JS events are resolved. When the semaphore is released, the wait callback is called and we process managed threadpool work.
  • If the semaphore times out, the timeout callback is called and we decide if there is any pending async IO (in the form of unsettled JS interop promises) and either schedule another semaphore wait or destroy the keepalive token allowing the thread to exit if the keepalive count reaches zero.
• Refactors LowLevelLifoSemaphore into two versions:
  • the "normal" version (created with the public constructor) that can be used with a synchronous Wait method,
  • and the "async wait" version (created with LowLevelLifoSemaphore.CreateAsyncWait()) that cannot use Wait but can use the callback-based PrepareAsyncWait method.
  • The common code for Wait and PrepareAsyncWait for managing the semaphore's counts is shared.
• Implements the underlying native LifoSemaphoreAsyncWait struct that shares some common code with the "normal" LifoSemaphore (via LifoSemaphoreBase).
  • To implement mono_lifo_semaphore_asyncwait_prepare_wait and mono_lifo_semaphore_asyncwait_release we use two emscripten APIs: emscripten_set_timeout to schedule a C callback to run on the current thread after a certain amount of time has passed; emscripten_dispatch_to_thread_async to allow one thread to queue a C callback to run on another pthread (note this depends on the main browser thread being responsive enough to relay JS messages from one thread to another).
  • When a thread starts to wait we create a wait queue entry and simultaneously start a timeout and return from prepare_wait.
  • To release the semaphore, we select a wait entry and dispatch its success callback to run on the waiting thread. When the success or timeout callbacks run, they unlink the wait entry (and the success callback cancels the timeout callback) and call the user-provided success/timeout functions.
  • (There is a case where the async success is queued but the timeout fires first that is handled by the timeout callback first checking if a success has been queued to run, too, in which case the timeout silently becomes a no-op)
• The managed threadpool worker keeps its pthread alive as long as the _js_owned_object_table is gc-handles.ts is non-empty: that is if there are any JS promises that will eventually callback to a C# Task on the current worker, even if the worker doesn't have any active threadpool work, it will keep the pthread alive until all the JS promises settle (and remove themselves from _js_owned_object_table.

Author:	lambdageek
Assignees:	lambdageek
Labels:	arch-wasm, area-VM-threading-mono
Milestone:	-

[kg]

this depends on the main browser thread being responsive enough to relay JS messages from one thread to another

This seems potentially a problem, IIRC it's common for stuff like Parallel.For to also use the main thread for computation when it forks off workers, or for applications to block the main thread waiting for workers to finish doing stuff. I remember at one point you were looking into setting up messageports between all of our workers, is there an obstacle to that or did it just not happen?

[lambdageek]

this depends on the main browser thread being responsive enough to relay JS messages from one thread to another

This seems potentially a problem, IIRC it's common for stuff like Parallel.For to also use the main thread for computation when it forks off workers, or for applications to block the main thread waiting for workers to finish doing stuff.

Don't do that. synchronous waits on the main thread continue to be problematic on the browser.

For an event driven UI app, I don't actually believe that it is common to block the main thread waiting for async work to complete, as that would lock up the UI.

I remember at one point you were looking into setting up messageports between all of our workers, is there an obstacle to that or did it just not happen?

Point to point messageports require bouncing through the main thread to set them up on demand. (creating a pthread requires proxying through the main thread, too). It seemed too expensive to set up all N^2 of them upfront (when we create each thread). So I wasn't entirely sure we need to do it, unless it's completely unavoidable.

It's not entirely accurate actually that the emscripten's emscripten_dispatch_to_thread_async requires the main thread to be in the JS event loop, always. because it's ferrying C data structures around, I believe the async work queues do get populated and pumped in other places as long as the worker thread is doing some emscripten syscalls. But if the worker thread is completely idle (or not interacting with emscripten at all), I believe the main thread needs to post it a message in order for the work to get dequeued and processed.

[lambdageek]

Should I slice this up into smaller PRs that might be easier to review? Something like:

• Refactoring of LifoSemaphore, but no implementation of AsyncWait
• Refactoring of ThreadPool.WorkerThread, but no implementation of the JS event loop version
• Adding the async wait lifo semaphore
• Adding the keepalive API and the JS eventloop ThreadPool.WorkerThread and updated smoketest

[kg]

Should I slice this up into smaller PRs that might be easier to review? Something like:

* Refactoring of LifoSemaphore, but no implementation of AsyncWait

* Refactoring of ThreadPool.WorkerThread, but no implementation of the JS event loop version

* Adding the async wait lifo semaphore

* Adding the keepalive API and the JS eventloop ThreadPool.WorkerThread and updated smoketest

Might be a good idea.

[lambdageek]

Created #84489 to summarize the work to be landed. Will close this PR in favor of the smaller pieces.

[lambdageek]

Closing this in favor of #84494 which is the last part of #84489