Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SelectMany with collection navigation throws #16989

Closed
ajcvickers opened this issue Aug 6, 2019 · 2 comments · Fixed by #17077
Closed

SelectMany with collection navigation throws #16989

ajcvickers opened this issue Aug 6, 2019 · 2 comments · Fixed by #17077
Assignees
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone

Comments

@ajcvickers
Copy link
Member

Tested on SQL Server.

Examples:

var petNames = context.People.SelectMany(x => x.Pets).ToList();
Unhandled exception. System.InvalidOperationException: SelectMany's collectionSelector was not NavigationExpansionExpression
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ProcessSelectMany(NavigationExpansionExpression source, MethodInfo genericMethod, LambdaExpression collectionSelector, LambdaExpression resultSelector)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryOptimizer.Visit(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.Main() in C:\Stuff\ThreeOhP7P\ThreeOhP7P\Program.cs:line 335
   at Program.<Main>()
var petNames = context.People.SelectMany(x => x.Pets.Select(y => y.Name)).ToList();
Unhandled exception. System.ArgumentException: The type or method has 2 generic parameter(s), but 3 generic argument(s) were provided. A generic argument must be provided for each generic parameter.
   at System.RuntimeType.SanityCheckGenericArguments(RuntimeType[] genericArguments, RuntimeType[] genericParamters)
   at System.Reflection.RuntimeMethodInfo.MakeGenericMethod(Type[] methodInstantiation)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ProcessSelectMany(NavigationExpansionExpression source, MethodInfo genericMethod, LambdaExpression collectionSelector, LambdaExpression resultSelector)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryOptimizer.Visit(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.Main() in C:\Stuff\ThreeOhP7P\ThreeOhP7P\Program.cs:line 335
   at Program.<Main>()

Workaround: evaluate the SelectMany on the client:

var petNames = context.People
    .Include(e => e.Pets)
    .AsEnumerable()
    .SelectMany(x => x.Pets.Select(y => y.Name)).ToList();

Full repo:

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; }
    public List<Pet> Pets { get; set; }
}

public class Pet
{
    public int Id { get; set; }
    public string Name { get; set; }
    public int PersonId { get; set; }
    public Person Person { get; set; }
}

public class BloggingContext : DbContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseSqlServer(@"Server=(localdb)\mssqllocaldb;Database=Test;ConnectRetryCount=0");

    public DbSet<Person> People { get; set; }
}

public class Program
{
    public static async Task Main()
    {
        using (var context = new BloggingContext())
        {
            context.Database.EnsureDeleted();
            context.Database.EnsureCreated();

            context.Add(new Person
            {
                Name = "brock",
                Pets = new List<Pet>
                    {
                        new Pet
                        {
                            Name = "max"
                        }
                    }
            });
            context.Add(new Person
            {
                Name = "dom",
                Pets = new List<Pet>
                    {
                        new Pet
                        {
                            Name = "mittens"
                        }
                    }
            });

            context.SaveChanges();
        }

        using (var context = new BloggingContext())
        {
            var petNames = context.People.SelectMany(x => x.Pets.Select(y => y.Name)).ToList();
        }
    }
}
@smitpatel
Copy link
Contributor

The overload of SelectMany which does not contain collectionSelector is not processed in navigation expansion yet since we don't translate it.

@ajcvickers
Copy link
Member Author

@smitpatel @maumar This is needed by IdentityServer, so we should get the fix into preview 9

@smitpatel smitpatel self-assigned this Aug 6, 2019
@ajcvickers ajcvickers added this to the 3.0.0 milestone Aug 6, 2019
@smitpatel smitpatel added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Aug 10, 2019
smitpatel added a commit that referenced this issue Aug 10, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

Add support for Cross Apply
Add Support for Outer Apply
Convert Cross Apply to Inner Join when possible
Convert Outer Apply to Left Join when possible
Add translation for DefaultIfEmpty
Add translation for both overloads of SelectMany
Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 10, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 12, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 13, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
@ajcvickers ajcvickers modified the milestones: 3.0.0, 3.0.0-preview9 Aug 21, 2019
@ajcvickers ajcvickers modified the milestones: 3.0.0-preview9, 3.0.0 Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants