CosmosDB: Nested embedded entities get rearranged on deletion

[sephnewman]

Overview

I'm working on a project that has a Cosmos DB container that contains multiple levels of nested embedded entities. We started to notice strange behavior that I've ultimately tracked down to an interaction between EF and CosmosDB. This behavior occurs when deleting middle level embedded entities from a CosmosDB record. When the middle level embedded entity is deleted, its inner child entities which we would expect to also be deleted are instead rearranged to point to a different middle level entity.

Example

We have three levels of entities:

• OuterEntity
• MiddleEntity
• InnerEntity

The OuterEntity owns many MiddleEntities and the MiddleEntity owns many InnerEntities

After we delete the first MiddleEntity, the one remaining MiddleEntity now contains the wrong InnerEntity

Code Example

This example is easily reproducible with a small program run against the local cosmos db emulator

using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.EntityFrameworkCore;

namespace CosmosDbBugDemo
{
    class Program
    {
        public class OuterEntity
        {
            public string OuterName { get; set; }
            public IList<MiddleEntity> MiddleEntities { get; set; }

            public override string ToString()
            {
                var outputStr = $"{OuterName}";
                foreach (var middleEntity in MiddleEntities)
                {
                    outputStr += $"\n\t{middleEntity.ToString()}";
                }
                return outputStr;
            }
        }

        public class MiddleEntity
        {
            public string MiddleName { get; set; }
            public IList<InnerEntity> InnerEntities { get; set; }

            public override string ToString()
            {
                var outputStr = $"{MiddleName}";
                foreach (var innerEntity in InnerEntities)
                {
                    outputStr += $"\n\t\t{innerEntity.ToString()}";
                }
                return outputStr;
            }
        }

        public class InnerEntity
        {
            public string InnerName { get; set; }

            public override string ToString()
            {
                return InnerName;
            }
        }

        // fixed key common to all local cosmos db emulators
        private const string LocalCosmosDbKey =
            "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";

        public class BugDemoContext : DbContext
        {
            protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
                => optionsBuilder.UseCosmos(
                    "https://localhost:8081",
                    LocalCosmosDbKey,
                    databaseName: "BugDemoDatabase");

            public DbSet<OuterEntity> OuterEntities { get; set; }

            protected override void OnModelCreating(ModelBuilder modelBuilder)
            {
                modelBuilder.Entity<OuterEntity>().HasKey(o => o.OuterName);
                modelBuilder.Entity<OuterEntity>()
                    .OwnsMany(o => o.MiddleEntities,
                        middleEntity =>
                        {
                            middleEntity.OwnsMany(m => m.InnerEntities);
                        });
            }
        }

        static void Main(string[] args)
        {
            // setup the DB
            using (var context = new BugDemoContext())
            {
                context.Database.EnsureDeleted();
                context.Database.EnsureCreated();
            }

            // add a test item to the DB
            using (var context = new BugDemoContext())
            {
                var outerEntity = new OuterEntity()
                {
                    OuterName = "Outer1",
                    MiddleEntities = new List<MiddleEntity>()
                    {
                        new MiddleEntity()
                        {
                            MiddleName = "Middle1",
                            InnerEntities = new List<InnerEntity>()
                            {
                                new InnerEntity()
                                {
                                    InnerName = "Inner1",
                                }
                            }
                        },
                        new MiddleEntity()
                        {
                            MiddleName = "Middle2",
                            InnerEntities = new List<InnerEntity>()
                            {
                                new InnerEntity()
                                {
                                    InnerName = "Inner2"
                                }
                            }
                        }
                    }
                };
                context.OuterEntities.Add(outerEntity);
                context.SaveChanges();
            }

            // get Outer1 from the DB and print its state
            Console.WriteLine("Initial State:");
            using (var context = new BugDemoContext())
            {
                var outer1 = context.OuterEntities.Find("Outer1");
                Console.WriteLine(outer1.ToString());
            }

            // delete Middle1 from the DB
            using (var context = new BugDemoContext())
            {
                var outer1 = context.OuterEntities.Find("Outer1");
                var middle1 = outer1.MiddleEntities.Single(m => m.MiddleName == "Middle1");
                outer1.MiddleEntities.Remove(middle1);
                context.Update(outer1);
                context.SaveChanges();
            }

            // get Outer1 from the DB and print its state
            Console.WriteLine("\nAfter Deletion:");
            using (var context = new BugDemoContext())
            {
                var outer1 = context.OuterEntities.Find("Outer1");
                Console.WriteLine(outer1.ToString());
            }
        }
    }
}

Code Output

The code above produces the following ouput to the console:

Initial State:
Outer1
        Middle1
                Inner1
        Middle2
                Inner2

After Deletion:
Outer1
        Middle2
                Inner1

Clearly, this is not expected. The output we would expect if the Middle1 record was properly deleted would be:

Outer1
        Middle2
                Inner2

EF provider and version information

I have tested and confirmed this behavior with two configurations to prove that this bug is present in both EF 5 and EF 6:

EF 5 config:

EF Core version: 5.0.13
Database provider: Microsoft.EntityFrameworkCore.Cosmos (5.0.13)
Target framework: netcoreapp3.1
Operating system: Windows 10 Pro 21H2 19044.1466
IDE: Microsoft Visual Studio Community 2019 (Version 16.11.3)

EF 6 config:

EF Core version: 6.0.1
Database provider: Microsoft.EntityFrameworkCore.Cosmos (6.0.1)
Target framework: net6.0
Operating system: Windows 10 Pro 21H2 19044.1466
IDE: Microsoft Visual Studio Community 2022 (64-bit) (Version 17.0.5)

[AndriySvyryd]

As a workaround detach all nested embedded entities for the entity being deleted right before SaveChanges():

foreach (var inner in middle1.InnerEntities)
{
	context.Entry(inner).State = EntityState.Detached;
}