NonSend resources off the main thread

[alice-i-cecile]

What problem does this solve or what need does it fill?

NonSend resources cannot be sent across threads, but our requirement is currently significantly stricter: systems that contain even one NonSend resource always run on the main thread.

This creates minor inefficiencies when working in a single world context, but causes much more serious problems in the context of multiple worlds.

If we are running two schedules in parallel, and each contains a system that operates on a distinct NonSend resource, under the current model, we must couple the two schedules to ensure that the main thread is free, even though no other coordination is required.

This limitation also blocks the ability to use more complex strategies for thread management, such as a dedicated input-polling thread.

What solution would you like?

Relax the limitation that NonSend resources must always be accessed on the main thread to the limitation that they are always accessed on the same thread.

In order to assign threads properly, we must evaluate the Schedule as a whole. As systems must run in exactly one thread, NonSend resources that are accessed in the same system must belong to the same thread.

This creates a particular challenge when it comes to exclusive systems, which access the contents of the entire World.

Option 1: Remove NonSend resources from the World

If NonSend resources are not part of the World, then exclusive systems must request them explicitly.
This avoids spurious conflicts, and allows us to find a thread-assignment that meets the constraints of the schedule.

However, this causes some serious limitations:

1. Systems which must operate on all NonSend resources associated with a World become dramatically harder to write and probably need to get moved into the runner. These should be extremely rare (perhaps restricted to editor-style inspection), but this is a noticeable ergonomics loss.
2. Scheduling access to NonSend resources that are accessed by more than one World requires coordination.
3. If and when we have the ability to dynamically add systems to the schedule (API for dynamic management of systems (adding and removing at runtime) #279), this strategy can result in users attempting to add systems that access two NonSend resources that are already assigned to different threads. The only possible solution is to error upon system insertion, which is frustrating at best.

Users could annotate systems with manual thread assignments (or hints) to work around this, but this is tedious and requires a very global view of both current and future schedule state. It is also rather unclear how this would work with respect to systems that have variable numbers of threads.

Note that this would resolve the problem fixed by #3519's turtles.

Option 2: Assign each World its own thread

This solution is straightforward, and allows zero-coordination execution of schedules on different worlds.
Furthermore, (assuming that schedules are bound to specific worlds) new systems will always be able to be added.

However:

• it blocks optimizations for scheduling systems involving NonSend resources within the same World
• not all threads are created equal: some OS processes must be created on the main thread, or some NonSend resources may have to be initialized in the runner
  • as a result, we must have two tiers of World: ones which are associated with the main thread (and thus must contend for access), and ones which are not (and can never access hypothetical MainThread resources)

[Guvante]

Systems which must operate on all NonSend resources associated with a World become dramatically harder to write and probably need to get moved into . These should be extremely

I think there is a typo here

[aevyrie]

Relax the limitation that NonSend resources must always be accessed on the main thread to the limitation that they are always accessed on the same thread.

How does this interact with things like winit, which requires some functions be run on the main thread (or maybe just the thread the event loop is on?). We might need an additional parameter for resources that rely on NonSend being on the main thread. See #4027.

[alice-i-cecile]

We might need an additional parameter for resources that rely on NonSend being on the main thread. See #4027.

Agreed. I think that there's fundamentally an important distinction between "main thread" and "fixed thread" that we need to tease apart.

[james7132]

#6534 technically adds support for NonSend resources from non-main threads, instead individually tracking where they were inserted from. However, the multithreaded scheduler currently does not support scheduling systems using these resources to anything other than the thread where the scope is running.

Moreover, in a previous discussion (see https://discord.com/channels/691052431525675048/692572690833473578/1047997793375096852 and #6552), looks like the general consensus is to move away from storing NonSends inside World. We may need to rethink how our strategy for this.