Construct and deconstruct entities to improve entity allocation #19451
Conversation
it's not exact, but it should be good enough.
ElliottjPierce
added
C-Feature
alice-i-cecile
changed the title
|
Does this fix #19012? I probably cannot test before the weekend. |
| let source_archetype = source_entity.archetype(); | ||
| let source_archetype = source_entity | ||
| .archetype() | ||
| .expect("Source entity must exist constructed"); |
There was a problem hiding this comment.
Are these expects actually more useful than an unwrap with a comment above?
,expect() wastes binary space with strings, often gives worse error messages, and don't get automatically updated as the error type changes.
There was a problem hiding this comment.
I ended up collapsing them together. Now, instead of "expect the entity exists" and "expect it is constructed" it is just one "expect the entity exists and is constructed". I think that strikes a good balance.
| pub(crate) unsafe fn declare( | ||
| &mut self, | ||
| row: EntityRow, | ||
| location: EntityIdLocation, |
There was a problem hiding this comment.
Changing this to new_location would help me understand how this function works a lot faster :)
There was a problem hiding this comment.
This is one of those times when I go "why didn't I think of that?". Now it's new_location and update_location.
There was a problem hiding this comment.
Judging that marking one line in github shows the three previous lines here, I think the suggestion was renaming the parameter, not the method. But it is still better now. 😁
There was a problem hiding this comment.
lol. I didn't notice that. 😂 But yeah, I think this is much better than before.
There was a problem hiding this comment.
Dramatically clearer than your initial attempts, and I'm sold on the functionality and core model. I'm also extremely pleased that we now have a number of ECS contributors who understand what's going on here. Thanks y'all!
I've left a few nits around docs and unwraps, but fundamentally nothing blocking. I'd also like to try and incorporate that lovely diagram into the entity module docs, but that's fine in follow-up.
The most important follow-up work for this is cleaning up commands.get_entity, but that can and should be done separately.
There was a problem hiding this comment.
What an Odyssee this is.
I understand this PR and while I approve it, I do have two notes:
- I think these names could use some bikeshedding, especially
EntityRowis confusing to me. I also think the docs could be more clear in places. - Secondly, I want to take a closer look at the seperation of responsibillities between EntityAllocator and Entities. Do they need to be seperate structs? Can the former be nested in the latter?
I'm worried that the average user will find this mighty confusing, but I say this with the caveat that I've been staring af this code for a couple hours and the average user might never have to, so keep that in mind.
| //! This column doesn't represents a component and is specific to the [`EntityRow`], not the [`Entity`]. | ||
| //! For example, one thing Bevy stores in this metadata is the current [`EntityGeneration`] of the row. | ||
| //! It also stores more information like the [`Tick`] a row was last constructed or destructed, and the [`EntityIdLocation`] itself. | ||
| //! For more information about what's stored here, see [`Entities`], Bevy's implementation of this special column. |
There was a problem hiding this comment.
I think this documentation is a good start, but I think a more direct route can be taken, along the lines of
"Entity spawning is done in two stages: First we create an entity id (alloc step), and then we construct it (add components and other metadata). The reason for this is that we need to be able to assign entity ids concurrently, while for construction we need exclusive (non-concurrent) access to the world. This leads to an entity having three states: It doesn't exist (unallocated), a id has been assigned but is not known to the rest of the world (null), and an entity that has been fully created (constructed)."
This, to me, seems like a much simpler mental model (even if it's partially incorrect). I think this documentation is fine for now, but it might be worth looking over again in a future PR.
There was a problem hiding this comment.
I like the clarity here. Yes, we need another docs pass, and book pass, etc. But for now, I adapted this into the storage docs. It was too good not to!
Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
Co-Authored-By: Trashtalk217 <24552941+Trashtalk217@users.noreply.github.com>
…ps://github.com/ElliottjPierce/bevy into Remove-entity-reserving/pending/flushing-system
@urben1680: Yes and no. No, it doesn't fix that. Technically, a freed entity does still exist. That is correct. The only requirement for an entity to exist is that the generation is up to date. This is intended behavior. For example, it could allow checking if the entity exists, and then constructing it. (In the particular case of a freed entity, this would cause errors when that entity is allocated, but Yes, it does add a |
I know, right?
Agreed. The more I think about it, the more "row" -> "index" renames make sense to me. And definitely more docs are needed in future PRs.
Yes, they can be together. In fact, originally, they were. But ultimately, they do very different things. There is no overlap of responsibility between them, and, more importantly, for remote reservation, the allocator needs to operate completely independently of As a side note, I see
I hear what you're saying here, but I don't think this will be an issue. Before, we had, |
Hm with two separate methods that distinction becomes more obvious to me which makes me lean more to the "yes" part of your answer. |
There was a problem hiding this comment.
This is only a cursory clean through the high level details. In particular, I haven't looked at the entities/mod.rs changes yet.
Conceptually, I like the idea and construction, especially since it gets rid of the pesky flushes that we were doing earlier.. However, this definitely comes with a lot more user-facing complexity that was otherwise siloed away, which, to me, violates Bevy's general policy of progressive disclosure. We may need to massage the API to either help check for some of errors statically, or in absence of that, move some of the new APIs into harder to reach for spaces to ensure users opt-in to the additional complexity..
I'll give this a more complete review before the end of October (will be unavailable for the majority of the next two weeks).
TODO on my end:
- Review the
entities/mod..rschanges. - Locally benchmark to validate the perf improvements seen in the PR description
- Run a quick check against bevy_asm_tests to check the codegen difference.
| /// All the entities to reuse. | ||
| /// This is a buffer, which contains an array of [`Entity`] ids to hand out. | ||
| /// The next id to hand out is tracked by `free_len`. | ||
| free: Vec<Entity>, |
There was a problem hiding this comment.
Does this need to store the generation too?
There was a problem hiding this comment.
Technically, no. This could just store row information. But then, Entities and EntitiesAllocator would need to be merged back together, which is not ideal. Separating them is necessary for remote entity reservation. You need to be able to hold the allocator without holding any reference to the world. But I think separating them is the job of this PR. It makes it very clear that the allocation stage is the job of EntitiesAllocator, and the construction stage is the job of Entities (and storage, etc).
| pub fn new_from_entities(queue: &'s mut CommandQueue, entities: &'w Entities) -> Self { | ||
| pub fn new_from_entities( | ||
| queue: &'s mut CommandQueue, | ||
| allocator: &'w EntitiesAllocator, |
There was a problem hiding this comment.
Is there any requirement that these two come from the same World?
There was a problem hiding this comment.
In theory, no. Though IIRC, it could cause a panic.
| /// | ||
| /// # Safety | ||
| /// | ||
| /// * Caller ensures that `queue` must outlive `'w` |
There was a problem hiding this comment.
It's been a while since I looked at this, but I don't see anything wrong about this as it is...
| #[inline(never)] | ||
| #[cold] | ||
| #[track_caller] | ||
| fn panic_no_entity(world: &World, entity: Entity) -> ! { |
There was a problem hiding this comment.
Was this removal necessary? This makes the error path no longer cold.
There was a problem hiding this comment.
I guess we could keep this, but panic should be cold anyway I think, so I don't think it would make a big difference. I think the old one had to be marked as cold because otherwise it would include entity_does_not_exist_error_details in the asm, but the new one doesn't have that problem. Still, its hard to predict how it compiles, so could be worth keeping anyway, Idk.
| /// // treat it as a normal entity | ||
| /// entity_access.despawn(); | ||
| /// ``` | ||
| pub fn spawn_null(&self) -> Entity { |
There was a problem hiding this comment.
Not a fan of this naming, but I can't really come up with an alternative off the top of my head.
| /// however, doing so is currently a bad idea as the allocator may hand out this entity row in the future, assuming it to be not constructed. | ||
| /// This would cause a panic. | ||
| /// | ||
| /// Manual construction is a powerful tool, but must be used carefully. |
There was a problem hiding this comment.
Like Alice's earlier comments, I'm a bit conflicted. This is rather advanced lifecycle manipulation that the overwhelming majority of Bevy devs should never need to touch. Moving these functions to somewhere that is intentionally harder to reach for seems like something we should do. Perhaps not in this PR, but definitely before this lands in a public release.
I'm also not sure if we should have a separate type for this, also wrapping UnsafeEntityCell, to encode this as a type-state instead of relying on runtime checks.
Co-authored-by: James Liu <contact@jamessliu.com>
|
Lots more good review from @james7132 ! From what I can tell, there are 3 outstanding concerns: First, does the performance and asm still hold up? Yes this needs more testing just because its been so long since this pr opened. Second, is this too complex for users? No. It doesn't increase complexity at all. It just changes alloc -> flush -> insert to alloc -> construct. If anything, that's simpler, albeit lower-level. The "api surface area" doesn't really change here IMO. Third, how should we name this and stop users from using this directly? IIRC (its been a few months) there has been discussion of putting "risky" or even What I can say is that I would oppose hiding the What I mean to say is that I don't see allocation and construction as a low-level primitive for the engine that just happens to be cleaner than before. I think these tools are just as useful to users as they are to the engine. As a small rant: Bevy markets itself as "simple," which sounds nice. But when you use a simple interface to do something fundamentally complex, it becomes really annoying! Want to load an asset? One line of code; easy. Want to do so with a tiny, simple loading screen? Time to bust out state machines, events, and all manner of chaos. I would much rather expose 100% of complexity to users and give them the tools to name and handle that complexity, than hide all that complexity behind simple but limiting interfaces. (I know this is not my decision to make, and for good reason, but I can't help but bring it up, as it's by far my biggest problem with Bevy and is what moved me from a Bevy user to a Bevy contributor.) On a completely different note, this pr is months old at this point, has a huge diff, and has so many comments and commits that it's starting to lag github (at least for me). Even if I could resolve all these merge conflicts, the result would be so different in implementation (same in concept of course) that all previous code review would probably need to be redone. When Bevy is ready for this, just let me know, and I will revive this I think on a new branch. And this time, I'll do my very best to keep the diff small, though that is not easy with a change so big. That said, I think getting the concept fully reviewed and agreed upon is still worth the work now, so I appreciate that very much. I feel like this pr has almost become more of an RFC than an implementation lol. |
11 participants
Objective
This is the next step for #19430 and is also convinient for #18670.
For context, the way entities work on main is as a "allocate and use" system. Entity ids are allocated, and given a location. The location can then be changed, etc. Entities that are free have an invalid location. To allocate an entity, one must also set its location. This introduced the need for pending entities, where an entity would be reserved, pending, and at some point flushed. Pending and free entities have an invalid location, and others are assumed to have a valid one.
This paradigm has a number of downsides: First, the entities metadata table is inseparable from the allocator, which makes remote reservation challenging. Second, the
Worldmust be flushed, even to do simple things, like allocate a temporary entity id. Third, users have little control over entity ids, only interacting with conceptual entities. This made things likeEntities::alloc_atclunky and slow, leading to its removal, despite some users still having valid need of it.So the goal of this PR is to:
Entitiesfrom entity allocation to make room for other allocators and resolvealloc_atissues.reserveandflushpatterns toallocandconstructpatterns.It is possible to break this up into multiple prs, as I originally intended, but doing so would require lots of temporary scaffolding that would both hurt performance and make things harder to review.
Solution
This solution builds on #19433, which changed the representation of invalid entity locations from a constant to
None.There's quite a few steps to this, each somewhat controversial:
Entities with no location
This pr introduces the idea of entity rows both with and without locations. This corresponds to entities that are constructed (the row has a location) and not constructed (the row has no location). When a row is free or pending, it is not constructed. When a row is outside the range of the meta list, it still exists; it's just not constructed.
This extends to conceptual entities; conceptual entities may now be in one of 3 states: empty (constructed; no components), normal (constructed; 1 or more components), or null (not constructed). This extends to entity pointers (
EntityWorldMut, etc): These now can point to "null"/not constructed entities. Depending on the privilege of the pointer, these can also construct or destruct the entity.This also changes how
Entityids relate to conceptual entities. AnEntitynow exists if its generation matches that of its row. AnEntitythat has the right generation for its row will claim to exist, even if it is not constructed. This means, for example, anEntitymanually constructed with a large index and generation of 0 will exist if it has not been allocated yet.Entitiesis separate from the allocatorThis pr separates entity allocation from
Entities.Entitiesis now only focused on tracking entity metadata, etc. The newEntitiesAllocatoronWorldmanages all allocations. This forcesEntitiesto not rely on allocator state to determine if entities exist, etc, which is convinient for remote reservation and needed for custom allocators. It also paves the way for allocators not housed within theWorld, makes some unsafe code easier since the allocator and metadata live under different pointers, etc.This separation requires thinking about interactions with
Entitiesin a new way. Previously, theEntitiesset the rules for what entities are valid and what entities are not. Now, it has no way of knowing. Instead, interaction withEntitiesare more like declaring some information for it to track than changing some information it was already tracking. To reflect this,sethas been split up intodeclareandupdate.Constructing and destructing
As mentioned, entities that have no location (not constructed) can be constructed at any time. This takes on exactly the same meaning as the previous
spawn_non_existent. It creates/declares a location instead of updating an old one. As an example, this makes spawning an entity now literately just allocate a new id and construct it immediately.Conversely, entities that are constructed may be destructed. This removes all components and despawns related entities, just like
despawn. The only difference is that destructing does not free the entity id for reuse. Between constructing and destructing, all needs foralloc_atare resolved. If you want to keep the id for custom reuse, just destruct instead of despawn! Despawn, now just destructs the entity and frees it.Destructing a not constructed entity will do nothing. Constructing an already constructed entity will panic. This is to guard against users constructing a manually formed
Entitythat the allocator could later hand out. However, public construction methods have proper error handling for this. Despawning a not constructed entity just frees its id.No more flushing
All places that once needed to reserve and flush entity ids now allocate and construct them instead. This improves performance and simplifies things.
Flow chart
(Thanks @ItsDoot)
Testing
Showcase
Here's an example of constructing and destructing
Future Work
Entitydoesn't always correspond to a conceptual entity.EntityWorldMut. There is (and was) a lot of assuming the entity is constructed there (was assuming it was not despawned).Performance
Benchmarks
This roughly doubles command spawning speed! Despawning also sees a 20-30% improvement. Dummy commands improve by 10-50% (due to not needing an entity flush). Other benchmarks seem to be noise and are negligible. It looks to me like a massive performance win!