Remote entity reservation

[ElliottjPierce]

Objective

fixes #18003

For assets as entities and components as entities, it is convinient to reserve an entity from an Arc, fully outside of the ECS. See here on discord for why.

It is probably possible to do both assets and components as entities without this, but we would have to eat extra complexity for that, and so would users making their own custom assets.

Solution

This design comes from here on discord.

Before this PR, EntityMeta is stored in a single Vec called meta. A pending list stores free entities, and an atomic cursor manages which pending entities are reserved and what new entities need to be created on meta.

All of that remains the same, but after this PR, an additional Vec<EntityMeta> called remote_entities exists. These indices are reversed from meta, so the index in remote_entities = u32::MAX - entity.index and visa versa. We only extend remote_entities based on an Arc<AtomicInt>. That counter represents remote_entities's length after the next flush. When we free one of these entities, we just add it to the pending list like normal, and the existing atomic cursor manages it.

Problems

I kinda had to tiptoe around Entity::PLACEHOLDER. The docs for that say it may exist, but with this, it will exist the first time the remote allocator is used. For now, I just start the generations at 2 instead of 1 to avoid accidental conflicts. This is easily changed if there are other ideas.

Alternatives

It is possible to create a paging/chunking system for Entities that would additionally allow entity partitioning. This design will simplify that if bevy chooses to add it in the future. In the meantime, this PR is much simpler than that.

Testing

We are not using the feature anywhere, so if current tests pass, we shouldn't have an issue.

However, it makes sense to add tests for this. If anyone has specific ideas for situations to test, I'd be more than happy to write them.

[andriyDev]

Can we add #18003 to this PR (as it should fix it)

[ElliottjPierce]

Also, I started this from #18190 since it is close to merging. Should merge that before this ideally.

[cart]

This is a reasonably workable solution, but it does have downsides:

1. Increases the cost of entity lookups (the fallback logic, less cache friendly as we're now hopping around two lists, etc).
2. Parallel entity reserving does not pull from the recycled entity pool. For parallel-heavy workloads (ex: assets that are continually loaded and unloaded), this could meaningfully eat into our id space.
3. It introduces a "meet in the middle" error case (which is not currently handled and solving this would introduce overhead). And given the "no parallel recycling behavior", the odds of hitting this go up.

If we're willing to forgo recycling (like we do in the current impl), we might be able to do away with the "two sided allocation" behavior to solve (1) and (3).

To do recycling, we'd probably need to introduce parallel queues (ex: what I do with our current Asset allocators) or introduce locking of some kind. Neither of which feels particularly good.

[andriyDev]

In regards to 2: I think we should just recreate the whole structure of pending + free_cursor for the parallel stuff. Whenever we flush, we can transfer the freed "parallel" entities to the RemoteEntityReservation. This will require us to lock on flush, and lock whenever reserving entities using RemoteEntityReservation. Personally I don't think this is a big issue - I think it's fine if remote entity reservation is slow. What is most important is non-remote entity reservation should be as fast as we can make it.

Edit: So I guess this is a parallel queue, but maintaining the "start from the back" idea.

[andriyDev]

Suggested change

	/// The counter for [`Self::remote_entities`]. This will equal it's length after flushing.
	/// The counter for [`Self::remote_entities`]. This will equal [`Self::remote_entities`] length after flushing.

[andriyDev]

If we want to get rid of this unsafe, I think just doing NonZero::new(2).unwrap() should work? Both new and unwrap are const.

[andriyDev]

Suggested change

	const REMOTE_FIRST_META: EntityMeta = const {
	let mut default = EntityMeta::EMPTY;
	default.generation = Self::REMOTE_FIRST_GENERATION;
	default
	};
	const REMOTE_FIRST_META: EntityMeta = EntityMeta {
	generation: Self::REMOTE_FIRST_GENERATION,
	..EntityMeta::EMPTY
	};

[andriyDev]

Suggested change

	/// Constructs a new [`RemoteEntityReserver`] for this [`Entities`].
	/// Get a [`RemoteEntityReserver`] for this [`Entities`].

The previous description sounded like it's creating like a new allocator or something to me.

[andriyDev]

Suggested change

	self.remote_reservations.0.fetch_and(0, Ordering::Relaxed);
	self.remote_reservations.0.store(0, Ordering::Relaxed);

[andriyDev]

I don't think these need to be u64? From what I understand, the normal cursor needs to be i64 because 1) we need to be able to handle 2^32 entities in pending, 2) we need to be able to handle 2^32 entities to be allocated that requires 33 bits to go +2^32 and -2^32. However, remote reservation only ever increments the cursor - it never goes down. So we only need to handle the +2^32, so we can stick to u32.