Tools for joining Query objects

[alice-i-cecile]

What problem does this solve or what need does it fill?

As users attempt to filter their data more finely in more complex applications, they may need to combine data from multiple queries in order to get precisely the data they need.

In the simple cases, adding more parameters and types to your query is the correct choice. But this fails when:

1. You need the data from the queries separately as well.
2. You want to express more complex compound queries, like "has component A or component B in this particular slot"

What solution would you like?

The standard relational database tools and/or set operations should be added, providing a set of (Query, Query) -> Query functions or methods.

The tools provided by Option<C> components already should be a heavy inspiration for the implementation and handling of the possibly ragged tables produced.

What alternative(s) have you considered?

We could provide for this behavior at the type level (see Or and the proposed Not query filters from #1471), allowing you to merge queries in complicated ways using the type system. This is less flexible and results in extremely complicated nested types, but does in theory mean we can restrict the data faster.

As a workaround, you could extract the list of entities in each query, perform the set operations on those entity lists, and then use query.get to extract the desired entities and data from each query separately.

Additional context

This came up in the context of #1627 with @BoxyUwU, discussing how to search for entities that have two parents. We can't sensibly have two Relation<ChildOf> types in the type parameter of a single Query argument without a great deal of API complication, but looking for intersections etc. of the same relation across multiple entities is a sensible and common operation.

[alice-i-cecile]

IMO the HashSet methods should get us virtually all of the way there, and are familiar and idiomatic to Rust. Implementing those features on queries should be fairly easy: don't be intimidated if you want to try your hand at this.

[bilsen]

I'd like to try this (a canidate for my first open-source contribution :) ). However, I have some questions about the proposed behaviour. What happens when two queries are conflicting? Should it be assumed that this never happens since systems cannot have two conflicting queries as parameters? Should a join between a Query<(&A, &B)> and Query<(&B, &C)> produce a Query<((&A, &B), (&B, &C))> or a Query<(&A, &B, &C)>? I would assume the first since the second would result in some type ambiguity depending on the order which the components are joined. Also, could you clarify how this would solve the problem about Relation<ChildOf>? How could an entity even have multiple parents, if an entity can only have one value for a given component?

[alice-i-cecile]

at happens when two queries are conflicting? Should it be assumed that this never happens since systems cannot have two conflicting queries as parameters?

I think this is correct, but I would be sure to add tests for it.

Should a join between a Query<(&A, &B)> and Query<(&B, &C)> produce a Query<((&A, &B), (&B, &C))> or a Query<(&A, &B, &C)>? I would assume the first since the second would result in some type ambiguity depending on the order which the components are joined.

I would prefer the latter, as it's much more ergonomic to work with and matches database joins more closely. There may be issues with this if we have &B and &mut B however 🤔 I suppose in that case you can downcast it all to an immutable &B, which I think is the correct trade-off to make here.

Also, could you clarify how this would solve the problem about Relation? How could an entity even have multiple parents, if an entity can only have one value for a given component?

Relations are an (experimental) exception to this rule: you can have multiple Relation<T> components of the same type, so long as they have different targets :) You can read more about them here: bevyengine/rfcs#18

[bilsen]

I would prefer the latter, as it's much more ergonomic to work with and matches database joins more closely. There may be issues with this if we have &B and &mut B however 🤔 I suppose in that case you can downcast it all to an immutable &B, which I think is the correct trade-off to make here.

Hmm. Yea I agree that it would be preferrable. I can't figure out any way to describe the type of that hypothetical join function though. From what i understand, it would require exponentially many implementations of the function (for all the different sizes of the pair of tuples, as well as placements of the similar component types in the tuples, if they are mutable, options etc. etc.). I dont see how to easily describe the type of the return value from the join function given its inputs.

[alice-i-cecile]

Right. I guess do it the first way then, and leave a TODO comment to change it once variadics are added to Rust.

[bilsen]

This was a bit more difficult than i initially thought. I have two main problems right now:

1. To construct the joined query i need a reference to a QueryState. I can (kinda) easily construct this in the join function (using some new try_init methods for fetch states that take a immutable reference to world) but I dont know where to store it so that i can give a reference to the new method of Query that has a long enough lifetime.
2. The joined query can have mutability conflicts with the queries it is made up with. Maybe this could be solved by instead moving the two queries and their join into a QuerySet somehow? (although that would be a bit less ergonomic)

Any ideas?

[alice-i-cecile]

To construct the joined query i need a reference to a QueryState. I can (kinda) easily construct this in the join function (using some new try_init methods for fetch states that take a immutable reference to world) but I dont know where to store it so that i can give a reference to the new method of Query that has a long enough lifetime.

Hmm, I'm not sure. @BoxyUwU want to take a crack at this one?

The joined query can have mutability conflicts with the queries it is made up with. Maybe this could be solved by instead moving the two queries and their join into a QuerySet somehow? (although that would be a bit less ergonomic)

This should be impossible in practice; our ownership model is a bit more sophisticated than Rust's since it works on a per-archetype level. I suspect the correct answer is to use unsafe here.