Query: Defer inline-ing owned navigation expansion till all joins are generated #26641
Conversation
… generated Issue: Expanding owned navigations in relational falls into 3 categories - Collection navigation - which always generates a subquery. The predicate is generated in LINQ to allow mutation of outer SelectExpression - Reference navigation sharing table - which picks column from same table without needing to add additional join. This only mutate the projection list for SelectExpression at subquery level - Reference navigation mapped to separate table - which generates additional joins. Generating join can cause push down on outer SelectExpression if it has facets (e.g. limit/offset). This pushdown causes owned expansion from category 2 to be outdated and invalid SQL since they get pushed down to subquery. While their relevant entity projection is updated inside SelectExpression we already inlined older object in the tree at this point. In order to avoid issue with outdated owned expansion, we defer actual inline-ing while processing owned navigations so that all navigations are expanded (causing any mutation on SelectExpression) before we inline values. This PR introduces DeferredOwnedExpansionExpression which remembers the source projection binding to SelectExpression (which will remain accurate through pushdown), and navigation chain to traverse entity projections to reach entity shaper for final owned navigation. This way, we get up-to-date information from SelectExpression after all joins are generated. We also find updated entity projection through binding after we generate a join if pushdown was required. Resolves #26592 The issue was also present in 5.0 release causing non-performant SQL rather than invalid SQL. During 5.0 we expanded owned navigations again while during client eval phase (which happens in customer scenario due to include). This caused to earlier owned reference to have correct columns. Though the entity projection for owner was changed due to pushdown so we didn't add latter reference navigation binding in correct entity projection causing us to expand it again during 2nd pass. The exact same issue doesn't occur for InMemory provider (due to slightly different implementation) though we should also make InMemory provider work this way, submitting in a different PR in case we need to cherry-pick this for patch.
|
Hello @smitpatel! Because this pull request has the p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (
|
|
Apologies, while this PR appears ready to be merged, I've been configured to only merge when all checks have explicitly passed. The following integrations have not reported any progress on their checks and are blocking auto-merge:
These integrations are possibly never going to report a check, and unblocking auto-merge likely requires a human being to update my configuration to exempt these integrations from requiring a passing check. Give feedback on thisFrom the bot dev teamWe've tried to tune the bot such that it posts a comment like this only when auto-merge is blocked for exceptional, non-intuitive reasons. When the bot's auto-merge capability is properly configured, auto-merge should operate as you would intuitively expect and you should not see any spurious comments. Please reach out to us at fabricbotservices@microsoft.com to provide feedback if you believe you're seeing this comment appear spuriously. Please note that we usually are unable to update your bot configuration on your team's behalf, but we're happy to help you identify your bot admin. |
| @@ -1206,7 +1204,23 @@ protected override Expression VisitExtension(Expression extensionExpression) | |||
| makeNullable); | |||
|
|
|||
| var joinPredicate = _sqlTranslator.Translate(Expression.Equal(outerKey, innerKey))!; | |||
| // Following conditions should match conditions for pushdown on outer during SelectExpression.AddJoin method | |||
There was a problem hiding this comment.
should we add a helper method on select expression for this?
There was a problem hiding this comment.
I thought about it. I had ran into similar for right side of join expression and eventually ended up using a bool out parameter since it was inside SelectExpression. If we expose a method right now then we tie into contract of supporting it/not breaking it ever. So I deferred to put actual method when we really need to expose is, especially if a provider needs to know.
|
Some day team will approve this PR. 😢 |
ghost
merged commit 
ajcvickers
changed the title
Issue: Expanding owned navigations in relational falls into 3 categories
In order to avoid issue with outdated owned expansion, we defer actual inline-ing while processing owned navigations so that all navigations are expanded (causing any mutation on SelectExpression) before we inline values.
This PR introduces DeferredOwnedExpansionExpression which remembers the source projection binding to SelectExpression (which will remain accurate through pushdown), and navigation chain to traverse entity projections to reach entity shaper for final owned navigation. This way, we get up-to-date information from SelectExpression after all joins are generated.
We also find updated entity projection through binding after we generate a join if pushdown was required.
Resolves #26592
The issue was also present in 5.0 release causing non-performant SQL rather than invalid SQL. During 5.0 we expanded owned navigations again while during client eval phase (which happens in customer scenario due to include). This caused to earlier owned reference to have correct columns. Though the entity projection for owner was changed due to pushdown so we didn't add latter reference navigation binding in correct entity projection causing us to expand it again during 2nd pass.
The exact same issue doesn't occur for InMemory provider (due to slightly different implementation) though we should also make InMemory provider work this way, submitting in a different PR in case we need to cherry-pick this for patch.