Support lazy and explicit loading from entities queried with NoTracking behavior

[JoesGab]

Explicit loading of a Reference does not populate the navigation property when QueryTrackingBehavior.NoTracking is set and AsTracking() is used as an override.

Explicit loading of a Collection does work as expected though.

Steps to reproduce

Minimalistic example for various combinations included. The issue arises only in the last assert statement of the second using block.

Program.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using Microsoft.EntityFrameworkCore;

namespace NoTrackingBug
{
    public class BloggingContext : DbContext
    {
        public DbSet<Blog> Blogs { get; set; }
        public DbSet<Post> Posts { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlite("Data Source=blogging.db");
        }
    }

    public class Blog
    {
        public int BlogId { get; set; }
        public string Url { get; set; }

        public List<Post> Posts { get; set; }

        public int OwnerId { get; set; }
        public Person Owner { get; set; }
    }

    public class Post
    {
        public int PostId { get; set; }
        public string Title { get; set; }

        public int BlogId { get; set; }
        public Blog Blog { get; set; }
    }

    public class Person
    {
        public int PersonId { get; set; }
        public string Name { get; set; }

        public List<Blog> OwnedBlogs { get; set; }
    }

    class Program
    {
        static void Main(string[] args)
        {
            // Setup the example
            using (var db = new BloggingContext())
            {
                if (!db.Blogs.Any())
                {
                    db.Blogs.Add(new Blog
                    {
                        Url = "http://blogs.msdn.com/adonet",
                        Posts = new List<Post>{
                            new Post { Title = "SomeTitle" }
                        },
                        Owner = new Person
                        {
                            Name = "Kobert"
                        }
                    });

                    db.SaveChanges();
                }
            }

            // Explicit load and .NoTracking => does not work for References
            using (var db = new BloggingContext())
            {
                db.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

                var blog = db.Blogs
                    .AsTracking()
                    .Single();

                db.Entry(blog)
                    .Collection(b => b.Posts)
                    .Load();

                db.Entry(blog)
                    .Reference(b => b.Owner)
                    .Load();

                Debug.Assert(blog.Posts != null);
                Debug.Assert(blog.Owner != null); // <== This will fail!
            }

            // Eager load and .NoTracking => does work
            using (var db = new BloggingContext())
            {
                db.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

                var blog = db.Blogs
                    .Include(b => b.Posts)
                    .Include(b => b.Owner)
                    .AsTracking()
                    .Single();

                Debug.Assert (blog.Posts != null);
                Debug.Assert (blog.Owner != null);
            }
            
            // Explicit load and .TrackAll => does work
            using (var db = new BloggingContext())
            {
                var blog = db.Blogs
                    .Single();

                db.Entry(blog)
                    .Collection(b => b.Posts)
                    .Load();

                db.Entry(blog)
                    .Reference(b => b.Owner)
                    .Load();

                Debug.Assert(blog.Posts != null);
                Debug.Assert(blog.Owner != null);
            }
        }
    }
}

Further technical details

EF Core version: 1.1.0 and 2.0.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer and Micrisoft.EntityFrameworkCore.Sqlite
Operating system: Windows 7 Enterprise
IDE: Visual Studio 2017 Enterprise and Visual Studio Code

[ajcvickers]

Note for triage: behavior is to throw in all cases, as it was in 2.0. Implementation is a bit tricky because it requires running a no-tracking query where the root entity is already materialized but not tracked. Proposing we push this to post 2.1.

[ajcvickers]

Triage: Leaving this in the backlog for now. Please vote for this feature if you need it.

[Suriman]

Both the Lazy Loading feature and the AsNoTraking feature are fundamental for implementing applications that use the WebRule tool from https://codeeffects.com/.

This tool allows to evaluate expressions on graphs of objects where these expressions are defined in a declarative way, which provides a lot of power to the users of an application. You can see examples in the section:
https://codeeffects.com/Business-Rule-Demo

The tool allows navigating through the graph of objects accessing properties and collections of objects and it is precisely in this feature that the EF Core and the support of Lazy Loading come into play. For WebRule it is transparent to access a property of an object that is itself a collection of other objects thanks to the Lazy Loading mechanism of EF Core. Since WebRule only requires access to the properties to read and evaluate them, it is not necessary for the data to be tracked, which greatly increases the performance if the Lazy Loading combined with the AsNoTraking is allowed.

Please consider giving priority to this issue as it is essential in many scenarios, including this one that we particularly use.

[Starkie]

@divega told us that if we wanted to try and solve it we should contact you on the issue, as it's important to our use case.
Could you please guide us a bit on how could it be done?

Thanks in advance

[ajcvickers]

@Starkie Can you be more explicit about what you are asking?

[divega]

@ajcvickers I met @Starkie and @Suriman recently. They expressed a lot of interest on creating a PR to address this issue. They are asking for guidance and details on what makes the issue tricky to fix.

[Starkie]

Sorry for being so ambiguous @ajcvickers 😅

As Diego said, we'd like to try to solve this issue. He commented that you may have some ideas on how could this be done, and might be able to offer us some guidance.

Sorry again, and thanks for your time.

[ajcvickers]

@Starkie The difficult aspect of this is changing the query pipeline so it can handle performing a no-tracking query with fixup when the root entity has already been materialized. This is the tricky bit that will need input from query people like @smitpatel and @maumar. I can't really give any kind of deep guidance on this without doing more digging and experimenting.

[smitpatel]

Filed #12208 to have better support to do fixup in QueryBuffer so that it can be reused for scenario like above.

[gokhanabatay]

Hi @ajcvickers, I dont understand why LazyLoading only available with "AsTracking()" queries, we are trying to migrate ef 7 from NHibernate. This is major issue since must support LazyLoading but "TrackAll" will kill application performance and memory. Is there any workaround to support lazy loading with non trackable queries?

[ajcvickers]

@dazbradbury If tracking is really too slow, then only workaround I can think of is to inject the DbContext into your entities (or your own proxies) and implement the behavior there.

[gokhanabatay]

@ajcvickers could you show us custom proxy sample, replacing ef core default proxy generator is it possible? #14554 , we have too much entity we cannot manuely add lazy initialization all of them. How to by pass EF Core LazyLoading exception for non tracking queries and allow lazy load?

[ajcvickers]

@gokhanabatay That's a fairly big job; I'm not sure I'm going to write a full example on this. If it's not something you feel like you can tackle, then probably best to wait for the feature to be implemented.

[MoazAlkharfan]

I think this made it so that proxies are being created and lazy loading is enabled even when ChangeTracker.LazyLoadingEnabled = false.

[ajcvickers]

@MoazAlkharfan Turning off lazy-loading does not disable proxy creation. Disabling proxy creation is tracked by #12895.