Including reference navigation and projecting collection navigation throws

[fschlaef]

When using .Include().ThenInclude() and selecting a collection, EF throws an InvalidOperationException with error EF.Property called with wrong property name.

Note that also using .Include() on the selected collection still throws.

The only "workaround" is to avoid .ThenInclude() which is not a solution for me (I need SomeCustomerDataFkNavigation)

Steps to reproduce

using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Debug;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace _18090Repro
{
    class Program
    {
        static void Main(string[] args)
        {
            using var context = new MyContext();

            context.Database.EnsureDeleted();
            context.Database.EnsureCreated();
            context.SomeCustomerData.Add(new SomeCustomerData { });
            context.SaveChanges();

            context.Customer.Add(new Customer() { SomeCustomerDataFk = 1 });
            context.Customer.Add(new Customer() { });
            context.SaveChanges();

            context.Invoice.Add(new Invoice { CustomerFk = 1 });
            context.SaveChanges();

            context.InvoiceLine.Add(new InvoiceLine { InvoiceFk = 1 });
            context.SaveChanges();

            var invoices = context.Invoice
                .Include(i => i.CustomerFkNavigation)
                    .ThenInclude(c => c.SomeCustomerDataFkNavigation)
                .Select(i => new Invoice()
                {
                    Id = i.Id,
                    CustomerFkNavigation = i.CustomerFkNavigation,
                    InvoiceLine = i.InvoiceLine // This throws
                })
                .ToList();
        }
    }

    public partial class MyContext : DbContext
    {
        public virtual DbSet<Customer> Customer { get; set; }
        public virtual DbSet<Invoice> Invoice { get; set; }
        public virtual DbSet<InvoiceLine> InvoiceLine { get; set; }
        public virtual DbSet<SomeCustomerData> SomeCustomerData { get; set; }

        private static readonly LoggerFactory Logger = new LoggerFactory(new[] { new DebugLoggerProvider() });

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            string connectionString = "Server=.;Database=Repro18090;Trusted_Connection=True;MultipleActiveResultSets=true";

            optionsBuilder.UseSqlServer(connectionString)
                .EnableSensitiveDataLogging()
                .UseLoggerFactory(Logger);
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Customer>(entity =>
            {
                entity.HasIndex(e => e.SomeCustomerDataFk);

                entity.Property(e => e.Id).HasColumnName("id");

                entity.Property(e => e.SomeCustomerDataFk).HasColumnName("someCustomerData_fk");

                entity.HasOne(d => d.SomeCustomerDataFkNavigation)
                    .WithMany(p => p.Customer)
                    .HasForeignKey(d => d.SomeCustomerDataFk)
                    .HasConstraintName("FK_Customer_SomeCustomerData");
            });

            modelBuilder.Entity<Invoice>(entity =>
            {
                entity.HasIndex(e => e.CustomerFk);

                entity.Property(e => e.Id).HasColumnName("id");

                entity.Property(e => e.CustomerFk).HasColumnName("customer_fk");

                entity.HasOne(d => d.CustomerFkNavigation)
                    .WithMany(p => p.Invoice)
                    .HasForeignKey(d => d.CustomerFk)
                    .OnDelete(DeleteBehavior.ClientSetNull)
                    .HasConstraintName("FK_Invoice_Customer");
            });

            modelBuilder.Entity<InvoiceLine>(entity =>
            {
                entity.Property(e => e.Id).HasColumnName("id");

                entity.Property(e => e.InvoiceFk).HasColumnName("invoice_fk");

                entity.HasOne(d => d.InvoiceFkNavigation)
                    .WithMany(p => p.InvoiceLine)
                    .HasForeignKey(d => d.InvoiceFk)
                    .OnDelete(DeleteBehavior.ClientSetNull)
                    .HasConstraintName("FK_InvoiceLine_Invoice");
            });

            modelBuilder.Entity<SomeCustomerData>(entity =>
            {
                entity.Property(e => e.Id).HasColumnName("id");
            });
        }
    }

    public partial class Customer
    {
        public Customer()
        {
            Invoice = new HashSet<Invoice>();
        }

        public int Id { get; set; }
        public int? SomeCustomerDataFk { get; set; }

        public virtual SomeCustomerData SomeCustomerDataFkNavigation { get; set; }
        public virtual ICollection<Invoice> Invoice { get; set; }
    }

    public partial class Invoice
    {
        public Invoice()
        {
            InvoiceLine = new HashSet<InvoiceLine>();
        }

        public int Id { get; set; }
        public int CustomerFk { get; set; }

        public virtual Customer CustomerFkNavigation { get; set; }
        public virtual ICollection<InvoiceLine> InvoiceLine { get; set; }
    }

    public partial class InvoiceLine
    {
        public int Id { get; set; }
        public int InvoiceFk { get; set; }

        public virtual Invoice InvoiceFkNavigation { get; set; }
    }

    public partial class SomeCustomerData
    {
        public SomeCustomerData()
        {
            Customer = new HashSet<Customer>();
        }

        public int Id { get; set; }

        public virtual ICollection<Customer> Customer { get; set; }
    }
}

Further technical details

EF Core version:
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: .NET Core 3.0
Operating system: Windows 10 x64
IDE: Visual Studio 2019 16.3.1

[guftall]

I get same error in ubuntu18 and EFCore 3.0.0
there was a workaround for me and i think this should work for you too, first convert your IQueryable to IEnumerable and then call Select method on that:

var invoices = context.Invoice
                .Include(i => i.CustomerFkNavigation)
                    .ThenInclude(c => c.SomeCustomerDataFkNavigation)
                .ToList()
                .Select(i => new Invoice()
                {
                    Id = i.Id,
                    CustomerFkNavigation = i.CustomerFkNavigation,
                    InvoiceLine = i.InvoiceLine
                });

[fschlaef]

@guftall This only works for small datasets as all you're doing is storing the entire table in memory, which would result in very poor performance with a lot of data.

[fschlaef]

@smitpatel confirmed fixed as of 3.1.0-preview3.19554.8 👍

[clement911]

How can such obvious bugs make it to RTM?
This has thrown our 2.2 to 3.0 migration completely off. Now do we have to park all of this migration and wait for 3.1?

[clement911]

Or can we use this preview package? I don't see it published on nuget. Is there another way of getting it?

[Suchiman]

@clement911 on nuget there are only preview releases, this is fixed for preview 3 but that's not out yet. It is available in daily builds for which you can find instructions here https://github.com/aspnet/AspNetCore/blob/master/docs/DailyBuilds.md

[clement911]

Got it thank you. Seems a bit risky for production so I guess I'll have to wait.
Will this fix be included in 3.1?

[heyligengregory]

I'm facing to the same problem. If it's included in 3.1, do you have any idea of the release date ?

[ErikEJ]

@heyligengregory 3.1 will come start of December 2019

[clement911]

Can you please include basic regression queries in the test suite? It's not like it's a super complicated query and it throws a runtime error.

[fschlaef]

@clement911 No offense but did you read the issue ? It clearly says that this issue is in the preview3 milestone, that it is closed as fixed (I even confirmed it in a message with the version detail), and if you look at the last commit (47f6cef) you will find tests called Including_reference_navigation_and_projecting_collection_navigation and Including_reference_navigation_and_projecting_collection_navigation_2 which target this issue.

This answers all your questions, and all the information was already there.

[clement911]

No offence taken. I'm a user of EF core and not a contributor. I'm not familiar with the inner workings of EF core or its testing code base (I wish I had time to).

I come from a user perspective and I just wanted to point out that it seems like such a severe regression on such a simple query should have been caught before EF Core 3.0 was released.
Is that not fair?

[ajcvickers]

@clement911 Yes, that's fair. That's why we run more than 50,000 tests on every build. However, the problem space for different queries is huge. Combine this with different models and model configurations and it gets even bigger. So sometimes things get missed. We do our best and continually strive to do better, but we make mistakes.

[clement911]

Fair enough. Apologies if I came out too strong but it was quite frustrating to have to abort our 3.0 migration after battling through all of the asp.net core migration issues because of this issue.

[Suchiman]

@clement911 with dotnet/announcements#139 or https://www.nuget.org/packages/Microsoft.EntityFrameworkCore/3.1.0-preview3.19554.8 it should be fixed now

[clement911]

Sweet thank you @Suchiman