Enhance & fix WorkerThreadPool

[RandomShaper]

A number of changes to the WorkerThreadPool, split into commits with specific goals:

2023-01-05: Heavily edited after squashing two commits together and greatly polishing some changes.

• WorkerThreadPool: Overhaul scheduling and synchronization

This commits rewrites the sync logic in a way that the use_system_threads_for_low_priority_tasks setting, which was added due to the lack of a cross-platform wait-for-multiple-objects functionality, can be removed (it's as if it was effectively hardcoded to false).

With the new implementation, we have the best of both worlds: threads don't have to poll, plus no bespoke threads are used.

In addition, regarding deadlock prevention, since not every possible case of wait-deadlock could be avoided, this commits removes the current best-effort avoidance mechanisms and keeps only a simple, pessimistic way of detection.

It turns out that the only current user of deadlock prevention, ResourceLoader, works fine with it and so every possible situation in resource loading is now properly handled, with no possibilities of deadlocking. There's a comment in the code with further details.

Lastly, a potential for load tasks never being awaited/disposed is cleared.

• WorkerThreadPool: Avoid deadlocks when CommandQueueMT is involved

This commit lets CommandQueueMT play nicely with the WorkerThreadPool to avoid non-progressable situations caused by an interdependence between both. While a command queue is being flushed, it allows the WorkerThreadPool to release
the lock of the former while tasks are being awaited so they can make progress in case they need in turn to post to the command queue.

• WorkerThreadPool: Avoid most runtime allocations

Just a little optimization.

NOTE:
With RID_Owner we could replace each pair of PagedAllocator and HashMap-of-ids-to-pointers. However, that would force us to expose RID as the task/group id, instead of int, which would break the API. Too bad. Let's wait until Godot 5.0.

---

Includes #86555.
Fixes #78790.

---

Testing appreciated. I'll keep testing it on different projects.

[fire]

The changes to the xml documentation are straightforward changes, but any idea why the asan and such is failing.

[RandomShaper]

Marking as Needs work because I'm rethinking one of the changes a bit, as I've found room for extra improvement.

[RandomShaper]

Now I'm happy with this.

[reduz]

Looks good, I like the new approach!

[clayjohn]

I just cancelled CI, the tests that run the unit tests appeared to be totally stuck. They had been running for 5 hours when I cancelled the workflow

[AThousandShips]

Sounds like a deadlock somewhere, suspect the threads aren't joined correctly

As a first step I'd suggest using RAII locking where possible to avoid problems there

[RandomShaper]

I've reproduced it locally. Thankfully the current tests stress the thing enough to make the issue apparent. When group tasks are involved in huge amounts, some low-prio tasks are kept in the queue with no threads realizing the need to claim them. Under investigation. 🔍

[RandomShaper]

OK. I've found and fixed the logic error.

[akien-mga]

Thanks!