Query: Contains on element coming from optional navigation doesn't get server-evaluated

[kierenj]

I'd like to help fix this, but if there's a workaround or suggested different way to express the same query that might work to get me around this in the short term, I'd be very grateful. Any workaround suggestions are welcome!

Steps to reproduce

Build a query with the following:

var set = (IQueryable<WidgetImage>)_context.Set<WidgetImage>();

// (there are some other filters here..)
 
// including this line, an "exclude any images where all of the textures IDs are excluded", breaks things
set = set.Where(i =>
    !i.UsedWidgets.Select(b =>
        b.WidgetsUsage.Brick.Texture.Id).All(t =>
            query.textures.ids.Contains(t)));

// (and here...)

var resultsProj = set.Select(w => new { w.Id }).Take(18);

Relevant entity models:

    public class WidgetImage
    {
        public Guid Id { get; set; }
        public List<ImageWidgetUsageLink> UsedWidgets { get; set; }
    }

    public class ImageWidgetUsageLink
    {
        public Guid ImageId { get; set; }
        public WidgetImage Image { get; set; }
        public Guid WidgetUsageId { get; set; }
        public WidgetUsage WidgetUsage { get; set; }
    }
    // config:
    e.HasKey(l => new { l.ImageId, l.WidgetUsageId });
    e.HasOne(l => l.Image).WithMany(i => i.UsedWidgets).IsRequired(true).HasForeignKey(l => l.ImageId);
    e.HasOne(l => l.WidgetUsage).WithMany(u => u.ImageLinks).IsRequired(true).HasForeignKey(l => l.WidgetUsageId);

    public class WidgetUsage
    {
        public Guid Id { get; set; }
        public Widget Widget { get; set; }
        public List<ImageWidgetUsageLink> ImageLinks { get; set; }
    }
    // config:
    e.HasOne(u => u.Widget).WithMany(b => b.Uses).IsRequired(true);

    public class Widget
    {
        public Guid Id { get; set; }
        public WidgetTexture Texture { get; set; }
        public List<WidgetUsage> Uses { get; set; }
    }
    // config:
    e.HasOne(b => b.Texture).WithMany(c => c.Widgets).IsRequired(false);

    public class WidgetTexture
    {
        public Guid Id { get; set; }
        public string Name { get; set; }
        public List<Widget> Widgets { get; set; }
    }

The issue

First issue is I get a warning that the Contains(b.WidgetsUsage.Brick.Texture.Id) cannot be translated to a store expression. I think this should be fine/possible - certainly it works with another very similar query clause. In that case I think the only difference is there's a SelectMany involved: replace one Texture with multiple Colours: set = set.Where(i => !i.UsedWidgets.SelectMany(b => b.WidgetUsage.Widget.Colours).Select(c => c.ColourId).All(c => query.colours.ids.Contains(c)));.

The second issue is, LINQ generation exception!

Exception:

Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.ni.dll

Additional information: variable 'b.WidgetUsage.Widget.Texture' of type 'MyProj.EntityModel.BrickTexture' referenced from scope '', but it is not defined

Stack trace:

   at System.Linq.Expressions.Compiler.VariableBinder.Reference(ParameterExpression node, VariableStorageKind storage)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitParameter(ParameterExpression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitUnary(UnaryExpression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitUnary(UnaryExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitLambda[T](Expression`1 node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.ExpressionVisitor.Visit(ReadOnlyCollection`1 nodes)
   at System.Linq.Expressions.Compiler.VariableBinder.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Compiler.LambdaCompiler.Compile(LambdaExpression lambda)
   at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateExecutorLambda[TResults]()
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateAsyncQueryExecutor[TResult](QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.System.Collections.Generic.IAsyncEnumerable<TResult>.GetEnumerator()
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.<ToListAsync>d__129`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.<ToArrayAsync>d__130`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at MyProj.Controllers.QueryController.<RunQuery>d__3.MoveNext() in C:\git\kierenj\myproj-platform\src\MyProj\Controllers\QueryController.cs:line 140

My LINQ expression DebugView:

.Call System.Linq.Queryable.Take(
    .Call System.Linq.Queryable.Select(
        .Call System.Linq.Queryable.Where(
            .Call System.Linq.Queryable.Where(
                .Constant<Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[MyProj.EntityModel.Image]>(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[MyProj.EntityModel.Image]),
                '(.Lambda #Lambda1<System.Func`2[MyProj.EntityModel.Image,System.Boolean]>)),
            '(.Lambda #Lambda2<System.Func`2[MyProj.EntityModel.Image,System.Boolean]>)),
        '(.Lambda #Lambda3<System.Func`2[MyProj.EntityModel.Image,<>f__AnonymousType12`1[System.Guid]]>)),
    18)

.Lambda #Lambda1<System.Func`2[MyProj.EntityModel.Image,System.Boolean]>(MyProj.EntityModel.Image $i) {
    .Call System.Linq.Enumerable.Any(
        .Call System.Linq.Enumerable.Select(
            $i.UsedWidgets,
            .Lambda #Lambda4<System.Func`2[MyProj.EntityModel.ImageWidgetUsageLink,MyProj.EntityModel.Widget]>),
        .Lambda #Lambda5<System.Func`2[MyProj.EntityModel.Widget,System.Boolean]>)
}

.Lambda #Lambda2<System.Func`2[MyProj.EntityModel.Image,System.Boolean]>(MyProj.EntityModel.Image $i) {
    !.Call System.Linq.Enumerable.All(
        .Call System.Linq.Enumerable.Select(
            $i.UsedWidgets,
            .Lambda #Lambda6<System.Func`2[MyProj.EntityModel.ImageWidgetUsageLink,System.Guid]>),
        .Lambda #Lambda7<System.Func`2[System.Guid,System.Boolean]>)
}

.Lambda #Lambda3<System.Func`2[MyProj.EntityModel.Image,<>f__AnonymousType12`1[System.Guid]]>(MyProj.EntityModel.Image $b)
{
    .New <>f__AnonymousType12`1[System.Guid]($b.Id)
}

.Lambda #Lambda4<System.Func`2[MyProj.EntityModel.ImageWidgetUsageLink,MyProj.EntityModel.Widget]>(MyProj.EntityModel.ImageWidgetUsageLink $b)
{
    ($b.WidgetUsage).Widget
}

.Lambda #Lambda5<System.Func`2[MyProj.EntityModel.Widget,System.Boolean]>(MyProj.EntityModel.Widget $b) {
    ($b.Shape).Ratio >= (System.Nullable`1[System.Double]).Constant<MyProj.Controllers.QueryController+<>c__DisplayClass3_0>(MyProj.Controllers.QueryController+<>c__DisplayClass3_0).minRatio
    && ($b.Shape).Ratio <= (System.Nullable`1[System.Double]).Constant<MyProj.Controllers.QueryController+<>c__DisplayClass3_0>(MyProj.Controllers.QueryController+<>c__DisplayClass3_0).maxRatio
}

.Lambda #Lambda6<System.Func`2[MyProj.EntityModel.ImageWidgetUsageLink,System.Guid]>(MyProj.EntityModel.ImageWidgetUsageLink $b)
{
    ((($b.WidgetUsage).Widget).Texture).Id
}

.Lambda #Lambda7<System.Func`2[System.Guid,System.Boolean]>(System.Guid $t) {
    .Call System.Linq.Enumerable.Contains(
        ((.Constant<MyProj.Controllers.QueryController+<>c__DisplayClass3_1>(MyProj.Controllers.QueryController+<>c__DisplayClass3_1).query).textures).ids,
        $t)
}

Further technical details

EF Core version: 1.1.0-preview1-*
Operating system: Windows 10
Visual Studio version: n/a (dotnet, 1.1 Preview 1)

[maumar]

Fixed in 8965903

[kierenj]

@maumar - thank you!