System.InvalidOperationException after upgraded to EF Core 3.1.2

[cobyzeng]

We are getting System.InvalidOperationException after upgraded to EF Core 3.1.2. It was working perfectly fine on EF Core 2.2 without any issue.

To setup the scene here:

In our application, we have orders, order contains multiple jobs and job has different job status. We use code first migration and utilised enumeration classes instead of enum types which is suggested by .NET microservices architecture e-book: https://docs.microsoft.com/en-us/dotnet/architecture/microservices/microservice-ddd-cqrs-patterns/enumeration-classes-over-enum-types

The symptom:

When adding a new job to the order that has already had a job whose job status is the same as the one about to be added, we will get the below exception:
System.InvalidOperationException: The instance of entity type 'JobStatus' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached. Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the conflicting key values.

Steps to reproduce

To simply things I have created a sample project to reproduce the problem: https://github.com/cobyzeng/EFCoreIssues

        [HttpPost("{id}/jobs")]
        public async Task<IActionResult> AddJobToOrder()
        {
            var order = await _orderRepository.Find(1);
            var job = new Job();
            order.AddJob(job);

            await _orderRepository.Complete();
            
            return Created("someroute", job);
        }

public class Order : Entity
    {
        public string OrderNumber { get; set; }
        public string CustomerName { get; set; }
        private readonly List<Job> _jobs;
        public IReadOnlyCollection<Job> Jobs => _jobs;

        public Order()
        {
            _jobs = new List<Job>();
        }

        public void AddJob(Job job)
        {
            var actionValidationResult = CanAddJob();
            if (!actionValidationResult)
            {
                throw new Exception("Some Reason");
            }

            _jobs.Add(job);
        }

        public bool CanAddJob()
        {
            return true;
        }
    }

 public class Job : Entity
    {
        public int OrderId { get; private set; }
        public string JobNumber { get; private set; }
        public JobStatus JobStatus { get; private set; }
        private int _jobStatusId;

        public Job()
        {
            JobNumber = $"Job_{Guid.NewGuid()}";
            UpdateStatusTo(JobStatus.Draft);
        }

        private void UpdateStatusTo(JobStatus status)
        {
            _jobStatusId = status.Id;
            JobStatus = status;
        }
    }

 public class JobStatus : Enumeration
    {
        public static readonly JobStatus Unassigned =
            new JobStatus(1, nameof(Unassigned).ToLowerInvariant(), "Unassigned");

        public static readonly JobStatus Assigned = new JobStatus(2, nameof(Assigned).ToLowerInvariant(), "Assigned");

        public static readonly JobStatus
            Completed = new JobStatus(3, nameof(Completed).ToLowerInvariant(), "Completed");

        public static readonly JobStatus Draft = new JobStatus(4, nameof(Draft).ToLowerInvariant(), "Draft");

        public string DisplayName { get; private set; }

        public JobStatus()
        {
        }

        public JobStatus(int id, string name, string displayName) : base(id, name)
        {
            DisplayName = displayName;
        }

        public static IEnumerable<JobStatus> List() => new[]
        {
            Unassigned,
            Assigned,
            Completed,
            Draft
        };
    }

 public class JobEntityTypeConfiguration : IEntityTypeConfiguration<Job>
    {
        public void Configure(EntityTypeBuilder<Job> builder)
        {
            builder.ToTable("jobs", OrderingContext.DEFAULT_SCHEMA);
            builder.HasIndex(x => x.JobNumber).IsUnique();
            builder.Property(p => p.JobNumber).HasMaxLength(100);
            builder.Property(p => p.OrderId).IsRequired();
            builder.Property<int>("_jobStatusId")
                .UsePropertyAccessMode(PropertyAccessMode.Field)
                .HasColumnName("JobStatusId")
                .IsRequired();
            builder.HasOne(o => o.JobStatus)
                .WithMany()
                .HasForeignKey("_jobStatusId");
        }
    }
    
    public class JobStatusEntityTypeConfiguration : IEntityTypeConfiguration<JobStatus>
    {
        public void Configure(EntityTypeBuilder<JobStatus> builder)
        {
            builder.ToTable("jobstatus", OrderingContext.DEFAULT_SCHEMA);

            builder.HasKey(c => c.Id);

            builder.Property(c => c.Name).HasMaxLength(100).IsRequired();
            builder.Property(c => c.DisplayName).HasMaxLength(100).IsRequired();

            builder.Property(o => o.Id)
                .ValueGeneratedNever()
                .IsRequired();
        }
    }
    public class OrderEntityTypeConfiguration : IEntityTypeConfiguration<Order>
    {
        public void Configure(EntityTypeBuilder<Order> builder)
        {
            builder.ToTable("orders", OrderingContext.DEFAULT_SCHEMA);
            builder.HasKey(x => x.Id);
            builder.HasIndex(x => x.OrderNumber).IsUnique();
            builder.Property(p => p.CustomerName).HasMaxLength(255).IsRequired(false);

            var jobsNavigation = builder.Metadata.FindNavigation(nameof(Order.Jobs));
            jobsNavigation.SetPropertyAccessMode(PropertyAccessMode.Field);

            builder.HasMany(c => c.Jobs)
                .WithOne()
                .HasForeignKey("OrderId")
                .OnDelete(DeleteBehavior.Cascade);
        }
    }

 public class OrderRepository : IOrderRepository
    {
        private readonly OrderingContext _context;

        public OrderRepository(OrderingContext context)
        {
            _context = context;
        }

        public async Task<Order> Find(int orderId)
        {
            var entity = await _context.Orders
                .Where(o => o.Id == orderId)
                .Include(o => o.Jobs)
                .ThenInclude(j => j.JobStatus)
                .FirstOrDefaultAsync();

            return entity;
        }

        public async Task Complete()
        {
           //_context.ChangeTracker.DetectChanges();
            await _context.SaveChangesAsync(true);
        }
    }

Exceptions:

System.InvalidOperationException: The instance of entity type 'JobStatus' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached. Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the conflicting key values.
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.IdentityMap`1.ThrowIdentityConflict(InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.IdentityMap`1.Add(TKey key, InternalEntityEntry entry, Boolean updateDuplicate)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.IdentityMap`1.Add(TKey key, InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.IdentityMap`1.Add(InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.StartTracking(InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState entityState, Boolean acceptChanges, Boolean modifyProperties, Nullable`1 forceStateWhenUnknownKey)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityGraphAttacher.PaintAction(EntityEntryGraphNode`1 node)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityEntryGraphIterator.TraverseGraph[TState](EntityEntryGraphNode`1 node, Func`2 handleNode)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityEntryGraphIterator.TraverseGraph[TState](EntityEntryGraphNode`1 node, Func`2 handleNode)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityGraphAttacher.AttachGraph(InternalEntityEntry rootEntry, EntityState targetState, EntityState storeGeneratedWithKeySetTargetState, Boolean forceStateWhenUnknownKey)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.NavigationCollectionChanged(InternalEntityEntry entry, INavigation navigation, IEnumerable`1 added, IEnumerable`1 removed)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.NavigationCollectionChanged(InternalEntityEntry entry, INavigation navigation, IEnumerable`1 added, IEnumerable`1 removed)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectNavigationChange(InternalEntityEntry entry, INavigation navigation)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.LocalDetectChanges(InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectChanges(IStateManager stateManager)
   at Microsoft.EntityFrameworkCore.ChangeTracking.ChangeTracker.DetectChanges()
   at Microsoft.EntityFrameworkCore.DbContext.TryDetectChanges()
   at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
   at Ordering.Repository.OrderRepository.Complete() in C:\Projects\CodeSnippet\Ordering\Repository\OrderRepository.cs:line 31
   at Ordering.Controllers.OrderController.AddJobToOrder() in C:\Projects\CodeSnippet\Ordering\Controllers\OrderController.cs:line 27
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Additional Information

If I put a breakpoint at line await _context.SaveChangesAsync(true); in OrderRepository and inspect the ChangeTracker, I can see the ChangedCount is 0, which i would expect not 0.

If I call _context.ChangeTracker.DetectChanges(); before saving change, the same exception gets thrown from there.

Further technical details

EF Core version: 3.1.2
Database provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Target framework: .NET Core 3.1
IDE: (e.g. Visual Studio 2019 16.3)

[AndriySvyryd]

This is probably a duplicate of #18007 as you are populating JobStatus in Job constructor.

EF Core also doesn't work well with singletons. As a workaround populate JobStatus in the getter and attach all instances of JobStatus before performing any queries.

[cobyzeng]

@AndriySvyryd

As a workaround populate JobStatus in the getter

Are you suggesting something like this? This still throws the same exception.

    public class Job : Entity
    {
        public int OrderId { get; private set; }
        public string JobNumber { get; private set; }

        public JobStatus JobStatus
        {
            get => JobStatus.From(_jobStatusId);
            private set => _jobStatusId = value.Id;
        }

        private int _jobStatusId;

        public Job()
        {
            JobNumber = $"Job_{Guid.NewGuid()}";
            _jobStatusId = JobStatus.Draft.Id;
        }
    }

and attach all instances of JobStatus before performing any queries.

i am not quite sure what you mean here. We are already doing .Include(o => o.Jobs).ThenInclude(j => j.JobStatus) in the query, is that what you mean?

Many thanks

[AndriySvyryd]

and attach all instances of JobStatus before performing any queries.

i am not quite sure what you mean here. We are already doing .Include(o => o.Jobs).ThenInclude(j => j.JobStatus) in the query, is that what you mean?

No, I mean

context.AttachRange(JobStatus.List());

And you can remove ThenInclude(j => j.JobStatus)

[cobyzeng]

Thanks @AndriySvyryd, Attaching Enumeration-based classes context.AttachRange(JobStatus.List()); before queries have fixed the problem.