Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Most cases of projecting a primitive collection don't work #30922

Closed
Tracked by #30731
ajcvickers opened this issue May 18, 2023 · 2 comments · Fixed by #31263
Closed
Tracked by #30731

Most cases of projecting a primitive collection don't work #30922

ajcvickers opened this issue May 18, 2023 · 2 comments · Fixed by #31263
Assignees
Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Milestone

Comments

@ajcvickers
Copy link
Member

using (var context = new SomeDbContext())
{
    context.Database.EnsureDeleted();
    context.Database.EnsureCreated();

    var days = await context.Walks
        .Select(e => e.DaysVisited.OrderBy(e => e.Day).ToList())
        .ToListAsync();
}

public class DogWalk
{
    public int Id { get; set; }
    public List<DateOnly> DaysVisited { get; private set; } = new();
}

public class SomeDbContext : DbContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseSqlServer(@"Data Source=(LocalDb)\MSSQLLocalDB;Database=AllTogetherNow")
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();

    public DbSet<DogWalk> Walks => Set<DogWalk>();
}
Unhandled exception. System.InvalidOperationException: Unable to translate a collection subquery in a projection since either parent or the subquery doesn't project necessary information required to uniquely identify it and correctly generate results on the client side. This can happen when trying to correlate
on keyless entity type. This can also happen for some cases of projection before 'Distinct' or some shapes of grouping key in case of 'GroupBy'. These should either contain all key properties of the entity that the operation is applied on, or only contain simple property access expressions.
   at Microsoft.EntityFrameworkCore.Query.SqlExpressions.SelectExpression.ApplyProjection(Expression shaperExpression, ResultCardinality resultCardinality, QuerySplittingBehavior querySplittingBehavior)
   at Microsoft.EntityFrameworkCore.Query.Internal.SelectExpressionProjectionApplyingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryTranslationPostprocessor.Process(Expression query)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerQueryTranslationPostprocessor.Process(Expression query)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass12_0`1.<ExecuteAsync>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\local\code\AllTogetherNow\Daily\Daily.cs:line 14
   at Program.<Main>(String[] args)
@roji roji self-assigned this May 18, 2023
@roji roji mentioned this issue May 18, 2023
34 tasks
@maumar maumar self-assigned this May 22, 2023
@roji
Copy link
Member

roji commented May 23, 2023

@maumar seems like you're interested, and you probably know more about this than me anyway :) Let me know if you want me to take a look too!

@ajcvickers ajcvickers added this to the 8.0.0 milestone May 24, 2023
@roji
Copy link
Member

roji commented Jul 8, 2023

Another example from #2981 (comment):

await using var ctx = new BlogContext();
await ctx.Database.EnsureDeletedAsync();
await ctx.Database.EnsureCreatedAsync();

var result = await ctx.Blogs
    .Select(address => string.Join(", ", new[] { address.Street1, address.City }.Where(s => s != "")))
    .ToListAsync();

public class BlogContext : DbContext
{
    public DbSet<Blog> Blogs { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        => optionsBuilder
            .UseSqlServer(@"Server=localhost;Database=test;User=SA;Password=Abcd5678;Connect Timeout=60;ConnectRetryCount=0;Encrypt=false")
            .LogTo(Console.WriteLine, LogLevel.Information)
            .EnableSensitiveDataLogging();
}

public class Blog
{
    public int Id { get; set; }
    public string? Street1 { get; set; }
    public string? City { get; set; }
}

@roji roji removed their assignment Jul 8, 2023
maumar added a commit that referenced this issue Jul 14, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 15, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 16, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 16, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 16, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 18, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 19, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 20, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 20, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Jul 20, 2023
maumar added a commit that referenced this issue Jul 20, 2023
… work

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
maumar added a commit that referenced this issue Jul 21, 2023
… work (#31263)

Problem was that for cases that compose on the collection of primitives and then project it, we need identifier to properly bucket the results. On SqlServer we can use hidden key column that is a result of OPENJSON - we need something that is unique so we can't use the value itself.
Fix is to add identifier to the SelectExpression based on OPENJSON and also modify the logic that figures out if we need key (and thus need to revert to OPENJSON without WITH clause). Also some minor fixes around nullability - when projecting element of a collection of primitives using OUTER APPLY/JOIN we must set the projection binding to nullable, as the value can always be null, in case of empty collection.

Fixes #30922
@ajcvickers ajcvickers modified the milestones: 8.0.0, 8.0.0-rc1 Aug 19, 2023
@ajcvickers ajcvickers modified the milestones: 8.0.0-rc1, 8.0.0 Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants