Exception when using inner entity with owned properties in the in-memory database

[AntonPervushin]

Getting an exception when using a queries with an owned property in an internal property.

System.InvalidCastException: 'Unable to cast object of type 'System.Guid' to type 'System.Nullable`1[System.DateTime]'.'

Looks like related to #23285
But that bug is realy fixed.
I took test model from that issue and get an exception in 5.0.0. And have no exception in 5.0.2 with that test model.

But I have an exception with my test model.

Test model

        public class Outer
        {
            public Guid Id { get; set; }
            public OwnedClass OwnedProp { get; set; }
            public Guid InnerId { get; set; }
            public Inner Inner { get; set; }
        }

        public class Inner
        {
            public Guid Id { get; set; }
            public OwnedClass OwnedProp { get; set; }
        }

        [Owned]
        public class OwnedClass
        {
            public DateTime At { get; set; }
        }

        public class TestContext : DbContext
        {
            public DbSet<Outer> Outers { get; set; }

            public DbSet<Inner> Inners { get; set; }

            public TestContext(DbContextOptions options)
                : base(options)
            {
            }
        }

Test case

            var options = new DbContextOptionsBuilder<TestContext>()
                .UseInMemoryDatabase("Test")
                .Options;
            var context = new TestContext(options);

            var inner = new Inner
            {
                Id = Guid.NewGuid(),
                OwnedProp = new OwnedClass{ At = DateTime.Now }
            };

            var outer = new Outer
            {
                Id = Guid.NewGuid(),
                OwnedProp = new OwnedClass { At = DateTime.Now },
                InnerId = inner.Id
            };

            context.Inners.Add(inner);
            context.Outers.Add(outer);
            context.SaveChanges();
            context.ChangeTracker.Clear();

            var criteria = DateTime.Now;

            var data = context.Outers
                .Where(x => x.OwnedProp.At >= criteria 
                            || x.Inner.OwnedProp.At >= criteria // Have no exception if comment this line
                            )
                .ToList();
        }

Stack trace:
at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext() at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable1.Enumerator.MoveNextHelper()
at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable1.Enumerator.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source)
at
...MyTest.cs:line 43

EF Core version: 5.0.1
Microsoft.EntityFrameworkCore.InMemory version 5.0.2
Database provider: Microsoft.EntityFrameworkCore.SqlServer/Microsoft.EntityFrameworkCore.InMemory
Target framework: (NET 3.1)
Operating system:
IDE: (Microsoft Visual Studio Professional 2019 16.7.6)

[ajcvickers]

Note for triage: regression from 3.1.

Logs:

dbug: Microsoft.EntityFrameworkCore.Model.Validation[10600]
      The property 'InnerId.Inner.OwnedProp#OwnedClass' was created in shadow state because there are no eligible CLR members with a matching name.
dbug: Microsoft.EntityFrameworkCore.Model.Validation[10600]
      The property 'OuterId.Outer.OwnedProp#OwnedClass' was created in shadow state because there are no eligible CLR members with a matching name.
info: Microsoft.EntityFrameworkCore.Infrastructure[10403]
      Entity Framework Core 5.0.2 initialized 'TestContext' using provider 'Microsoft.EntityFrameworkCore.InMemory' with options: SensitiveDataLoggingEnabled StoreName=Test
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10806]
      Context 'TestContext' started tracking 'Inner' entity with key '{Id: 10b76e92-e6a3-48c5-9cba-fc786801269f}'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10806]
      Context 'TestContext' started tracking 'OwnedClass' entity with key '{InnerId: 10b76e92-e6a3-48c5-9cba-fc786801269f}'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10806]
      Context 'TestContext' started tracking 'Outer' entity with key '{Id: 0e5777de-c558-4025-83da-985b3c2b1c83}'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10806]
      Context 'TestContext' started tracking 'OwnedClass' entity with key '{OuterId: 0e5777de-c558-4025-83da-985b3c2b1c83}'.
dbug: Microsoft.EntityFrameworkCore.Update[10004]
      SaveChanges starting for 'TestContext'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10800]
      DetectChanges starting for 'TestContext'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10801]
      DetectChanges completed for 'TestContext'.
info: Microsoft.EntityFrameworkCore.Update[30100]
      Saved 4 entities to in-memory store.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10807]
      The 'Inner' entity with key '{Id: 10b76e92-e6a3-48c5-9cba-fc786801269f}' tracked by 'TestContext' changed state from 'Added' to 'Unchanged'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10807]
      The 'Outer' entity with key '{Id: 0e5777de-c558-4025-83da-985b3c2b1c83}' tracked by 'TestContext' changed state from 'Added' to 'Unchanged'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10807]
      The 'OwnedClass' entity with key '{InnerId: 10b76e92-e6a3-48c5-9cba-fc786801269f}' tracked by 'TestContext' changed state from 'Added' to 'Unchanged'.
dbug: Microsoft.EntityFrameworkCore.ChangeTracking[10807]
      The 'OwnedClass' entity with key '{OuterId: 0e5777de-c558-4025-83da-985b3c2b1c83}' tracked by 'TestContext' changed state from 'Added' to 'Unchanged'.
dbug: Microsoft.EntityFrameworkCore.Update[10005]
      SaveChanges completed for 'TestContext' with 4 entities written to the database.
dbug: Microsoft.EntityFrameworkCore.Query[10111]
      Compiling query expression:
      'DbSet<Outer>()
          .Where(x => x.OwnedProp.At >= __criteria_0 || x.Inner.OwnedProp.At >= __criteria_0)'
dbug: Microsoft.EntityFrameworkCore.Query[10112]
      Including navigation: 'Outer.OwnedProp'.
dbug: Microsoft.EntityFrameworkCore.Query[10107]
      Generated query execution expression:
      'queryContext => new QueryingEnumerable<Outer>(
          queryContext,
          InMemoryShapedQueryCompilingExpressionVisitor.Table(
              queryContext: queryContext,
              entityType: EntityType: Outer)
              .Join(
                  inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                      queryContext: queryContext,
                      entityType: EntityType: Inner),
                  outerKeySelector: valueBuffer => (Nullable<Guid>)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 1,
                      property: Property: Outer.InnerId (Guid) Required FK Index),
                  innerKeySelector: valueBuffer => (Nullable<Guid>)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Inner.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  resultSelector: (outer, inner) => new ValueBuffer(new object[]
                  {
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer,
                          index: 0,
                          property: Property: Outer.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer,
                          index: 1,
                          property: Property: Outer.InnerId (Guid) Required FK Index),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: inner,
                          index: 0,
                          property: Property: Inner.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd)
                  }))
              .GroupJoin(
                  inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                      queryContext: queryContext,
                      entityType: EntityType: Outer.OwnedProp#OwnedClass),
                  outerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Outer.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  innerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Outer.OwnedProp#OwnedClass.OuterId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                  resultSelector: (outer, inner) => new TransparentIdentifier<ValueBuffer, IEnumerable<ValueBuffer>>(
                      Outer = outer,
                      Inner = inner
                  ))
              .SelectMany(
                  collectionSelector: collection => collection.Inner
                      .DefaultIfEmpty(new ValueBuffer(new object[]
                      {
                          null,
                          null
                      })),
                  resultSelector: (outer, inner) => new ValueBuffer(new object[]
                  {
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 0,
                          property: Property: Outer.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 1,
                          property: Property: Outer.InnerId (Guid) Required FK Index),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 2,
                          property: Property: Inner.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<Guid>>(
                          valueBuffer: inner,
                          index: 0,
                          property: Property: Outer.OwnedProp#OwnedClass.OuterId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                          valueBuffer: inner,
                          index: 1,
                          property: Property: Outer.OwnedProp#OwnedClass.At (DateTime) Required)
                  }))
              .GroupJoin(
                  inner: InMemoryShapedQueryCompilingExpressionVisitor.Table(
                      queryContext: queryContext,
                      entityType: EntityType: Inner.OwnedProp#OwnedClass),
                  outerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 2,
                      property: Property: Inner.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  innerKeySelector: valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Inner.OwnedProp#OwnedClass.InnerId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                  resultSelector: (outer, inner) => new TransparentIdentifier<ValueBuffer, IEnumerable<ValueBuffer>>(
                      Outer = outer,
                      Inner = inner
                  ))
              .SelectMany(
                  collectionSelector: collection => collection.Inner
                      .DefaultIfEmpty(new ValueBuffer(new object[]
                      {
                          null,
                          null
                      })),
                  resultSelector: (outer, inner) => new ValueBuffer(new object[]
                  {
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 0,
                          property: Property: Outer.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 1,
                          property: Property: Outer.InnerId (Guid) Required FK Index),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<Guid>>(
                          valueBuffer: outer.Outer,
                          index: 3,
                          property: Property: Outer.OwnedProp#OwnedClass.OuterId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                          valueBuffer: outer.Outer,
                          index: 4,
                          property: Property: Outer.OwnedProp#OwnedClass.At (DateTime) Required),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                          valueBuffer: outer.Outer,
                          index: 2,
                          property: Property: Inner.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<Guid>>(
                          valueBuffer: inner,
                          index: 0,
                          property: Property: Inner.OwnedProp#OwnedClass.InnerId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                      (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                          valueBuffer: inner,
                          index: 1,
                          property: Property: Inner.OwnedProp#OwnedClass.At (DateTime) Required)
                  }))
              .Where(valueBuffer => ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                  valueBuffer: valueBuffer,
                  index: 4,
                  property: Property: Outer.OwnedProp#OwnedClass.At (DateTime) Required) >= (Nullable<DateTime>)InMemoryExpressionTranslatingExpressionVisitor.GetParameterValue<DateTime>(
                  queryContext: queryContext,
                  parameterName: "__criteria_0") || ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                  valueBuffer: valueBuffer,
                  index: 6,
                  property: Property: Inner.OwnedProp#OwnedClass.At (DateTime) Required) >= (Nullable<DateTime>)InMemoryExpressionTranslatingExpressionVisitor.GetParameterValue<DateTime>(
                  queryContext: queryContext,
                  parameterName: "__criteria_0"))
              .Select(valueBuffer => new ValueBuffer(new object[]
              {
                  (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 0,
                      property: Property: Outer.Id (Guid) Required PK AfterSave:Throw ValueGenerated.OnAdd),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<Guid>(
                      valueBuffer: valueBuffer,
                      index: 1,
                      property: Property: Outer.InnerId (Guid) Required FK Index),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<Guid>>(
                      valueBuffer: valueBuffer,
                      index: 2,
                      property: Property: Outer.OwnedProp#OwnedClass.OuterId (no field, Guid) Shadow Required PK FK AfterSave:Throw),
                  (object)ExpressionExtensions.ValueBufferTryReadValue<Nullable<DateTime>>(
                      valueBuffer: valueBuffer,
                      index: 3,
                      property: Property: Outer.OwnedProp#OwnedClass.At (DateTime) Required)
              })),
          Func<QueryContext, ValueBuffer, Outer>,
          TestContext,
          False
      )'
Unhandled exception. failSystem.InvalidCastException: Unable to cast object of type 'System.Guid' to type 'System.Nullable`1[System.DateTime]'.
   at lambda_method101(Closure , ValueBuffer )
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
   at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Program.Main() in C:\Stuff\AllTogetherNow\FiveOh\Program.cs:line 89
   at Program.<Main>()
: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'TestContext'.
      System.InvalidCastException: Unable to cast object of type 'System.Guid' to type 'System.Nullable`1[System.DateTime]'.
         at lambda_method101(Closure , ValueBuffer )
         at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
         at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
         at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()
      System.InvalidCastException: Unable to cast object of type 'System.Guid' to type 'System.Nullable`1[System.DateTime]'.
         at lambda_method101(Closure , ValueBuffer )
         at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
         at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNextHelper()
         at Microsoft.EntityFrameworkCore.InMemory.Query.Internal.InMemoryShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.Enumerator.MoveNext()

Process finished with exit code -532,462,766.

[smitpatel]

Removing this from backlog for consider not patching this.
Work-around:
If we revert the fix for #23285 then it works correctly. Using AppContext switch should be fine way to go.
Technical challenge:

• This issue right now is more complex case of Misinterpreted expression with owned entities #20729 which we first fixed Misinterpreted expression with owned entities #20729 in Don't move my cheese! #22202 which we reverted and fixed differently in [5.0.1] InMemory: Copy correct value buffer data when doing LeftJoin #23292.
• We generate GJ-DIE-SM for LeftJoin in InMemory database. When doing this for weak navigation expansion, there are 2 major case which arises
  • Derived types generate value buffer expressions in form of a == "Derived" ? a.Property : null which we need to integrate and generate new value buffer read expression. What we did in [5.0.1] InMemory: Copy correct value buffer data when doing LeftJoin #23292
  • If earlier result is join of 2 entities and both of them have owned entities expanded in same lambda then re-generating new value buffer expressions changes indexes but this change of indexes causes first expansion to have outdated indexes causing above error.
• So we need to preserve indexes and we also need to re-generate new ones. Both of the cases are not easily separated (what if TPH & double owned expansion is in same case).

Even if we manage to come with set of conditions to play with indexes to fix this particular issue, we may cause another regression in some other case. Given there is work-around possible here and risk is high, we should avoid patching this and rather fix #21677 for 6.0 to fix these issues more robustly.

[AntonPervushin]

@smitpatel, thank you for your investigation of the problem.
If I understand correctly, the workaround you are talking about is ef-side. And time of it completion is not known.
Maybe you can suggest a client-side workaround?
We moved from ef 2.2 to ef 5, fixed all breaking changes and now we have no problems with SqlProvider.
But a lot of part of unit tests are failed and we can't test our business logic.

[smitpatel]

Client side work-around is to call following line right before the query

AppContext.SetSwitch("Microsoft.EntityFrameworkCore.Issue23285", true);

If the app is working for you with calling above code only once at the startup then that is also fine. If there are other queries which would break with it then you can set the switch to false again after your above query has executed.

[AntonPervushin]

If I set

AppContext.SetSwitch("Microsoft.EntityFrameworkCore.Issue23285", true);

then instead of

System.InvalidCastException : Unable to cast object of type 'System.String' to type 'System.Nullable`1[System.DateTime]'.

I get

System.NullReferenceException : Object reference not set to an instance of an object.

in my unit tests

[ajcvickers]

@AntonPervushin Given that setting the AppContext switch works with the repro code posted above, this means that there must be something different about the issue in your production code. Could you update the code you posted so that it fails in the same way as your production code when setting the AppContext switch?

Also, could you post the full stack trace of the failure you are getting with the AppContext switch set?

[AntonPervushin]

@ajcvickers I can not reproduce that with test model yet.
There might be some problems with my unit tests fixtures.
I will be look further.