Receiving: "System.NotSupportedException : Collection was of a fixed size." when not using a fixed-size collection.

[rcbellamy]

I am receiving an error from EF Core complaining that a collection was of a fixed size when I try to have multiple entities refer to the same entity as one of their values. I have done absolutely nothing to suggest that there should be any such constraint. I have not specified any kind of relationship that would limit the number of entities that can refer to the entity in question. I have not configured any primary or foreign key that would result in such a restriction. When I examine the database that results from running the code, I see that an extra column has been added to the database that has the effect of creating such a restriction, however I can find no basis in my code for the existence of that column. The column should not exist in the database, and I have not been able to track down the aspect of my code that results in its creation. I have tried to isolate the aspect that causes it to exist by commenting out portions and telling EF Core to ignore what needs to be ignored in order to comment out a few lines of code in OnModelCreating( ), but no matter what I comment out, the column remains.

Project Background

In this project, I have re-implemented Dictionary<TKey,TValue> using a collection of key-value pairs, in order to be able to store dictionaries using EF Core. If EF Core does in fact provide direct support for dictionaries, the documentation for this is far from adequate and without the documentation the feature might as well not be there. StackOverflow seemed to think it could not be done. Right now there is almost nothing else to this project. An object should be usable as a value in multiple dictionaries. The object is a property of a key-value pair, and each key-value pair is associated with one and only one dictionary. However, I cannot figure out what I have done that gives EF Core the impression that each object used as the value for a "Value" property of a key-value pair should be associated with a particular dictionary itself. For some reason, EF Core is under this impression.

This project would be extremely trivial if I were writing C# code that issued SQL commands to a database, but the world seems convinced that EF Core provides enough value to be worth using instead of the direct SQL approach, so no one seems to write about the direct SQL approach much anymore. Unfortunately, I have also found a dearth of adequate tutorial writing on EF Core as well (no, blogs absolutely do not count; can someone please recommend a book that would have been titled EF Core in a Nutshell if published by O'Reilly?). However, this is a really simple project, and it seems to me quite obvious how every aspect of this project, as currently written, should behave. Whatever is behaving in an unexpected way is so completely non-obvious that I really don't know where to look. The "Download PDF" option at https://docs.microsoft.com/en-us/ef/core/ is about 800 pages before the API reference. I suppose I can start reading that cover-to-cover.

This zip file, EFCoreProject.zip, contains the project as well as the unit test project. You will see some lines commented out relating to a NumberLib namespace and a Number data type. Number is simply a number data type that stores a numerator and a denominator as big integers, but it has been commented out and replaced with double so that you can run this project and reproduce the result without my needing to include unrelated files. The zip file also contains the database file that results when the code section indicated under "Failing Unit Test[,]" below, is commented out.

DbContext

The code for the DbContext subclass is as follows:

using CentralFunctionality.Data;
using Microsoft.EntityFrameworkCore;

namespace CentralTests.Data;

public class TestDbContextTwo : DbContext
{
    public TestDbContextTwo( DbContextOptions<TestDbContextTwo> options ) : base( options )
    {
        //
    }

    public DbSet<DataObject> DataObjects => Set<DataObject>( );
    
    protected override void OnModelCreating( ModelBuilder modelBuilder )
    {
        modelBuilder.Entity<DataObject>( )
                    .HasDiscriminator<string>( "DataObjectSubclass" )
                    .HasValue<DataValue>( nameof(DataValue) )
                    .HasValue<DataDictionary<string, DataObject>>( "DataDictionary" );
        
        modelBuilder.Entity<DataKeyValuePair<string, DataObject>>( )
                    .HasOne<DataDictionary<string, DataObject>>( )
                    .WithMany( d => d.KeyValuePairs )
                    .HasForeignKey( nameof(DataKeyValuePair<string, DataObject>.DictionaryId) );

        modelBuilder.Entity<DataKeyValuePair<string, DataObject>>( )
                    .HasKey( kvp => new { kvp.DictionaryId, kvp.Key } );

        modelBuilder.Entity<DataValue>( )
                    .Property( dv => dv.Value )
                    .HasConversion( v  => DataValue.Serialize( v ), 
                                    sv => DataValue.Deserialize( sv ) );

        modelBuilder.Entity<DataKeyValuePair<string, DataObject>>( )
                    .Property( kvp => kvp.Key )
                    .HasColumnName( "key" );

        modelBuilder.Entity<DataKeyValuePair<string, DataObject>>( )
                    .HasOne<DataObject>( kvp => kvp.Value )
                    .WithMany( );
        
        base.OnModelCreating( modelBuilder );
    }
}

Failing Unit Test

As noted in the code, if you simply comment out the part where the same value is stored in a second dictionary, the test passes. The screenshot below the code shows a portion of the database that results when the indicated section is commented out.

    [Test]
    public void Test2( )
    {
        //Database Setup
        var connection   = new SqliteConnection( "Data Source = testTwo.db" );
        var options      = new DbContextOptionsBuilder<TestDbContextTwo>( ).UseSqlite( connection ).Options;
        var dbContextTwo = new TestDbContextTwo( options );
        dbContextTwo.Database.EnsureDeleted( );
        var created = dbContextTwo.Database.EnsureCreated( );

        //Arrange
        var numberBook = new DataDictionary<string, DataObject>( );
        numberBook["one-half"] = new DataValue { Value = 0.5 };//new Number( 1, 2 ) };
        numberBook["two-thirds"] = new DataValue { Value = 0.667  };//new Number( 2, 3 ) };
        
        var megaBook = new DataDictionary<string, DataObject>( );
        megaBook["Numbers"] = numberBook;
        megaBook["Text"] = new DataValue { Value = "This is a simple sentence." };

        dbContextTwo.DataObjects.Add( megaBook );
        
        #region Comment this out and the test passes
        var extraHolder = new DataDictionary<string, DataObject>( );
        extraHolder["duplicate"] = numberBook;
        
        dbContextTwo.DataObjects.Add( extraHolder );
        #endregion
        
        dbContextTwo.SaveChanges( );
        //Act
        numberBook = null;
        megaBook = null;
        dbContextTwo.Dispose();
        dbContextTwo = new TestDbContextTwo( options );

        //Assert, well sort of
        // foreach( var bk in dbContextTwo.DataObjects )
        // {
        //     TestContext.WriteLine( $"{bk.Id}" );
        // }

        //Database Cleanup
        dbContextTwo.Dispose( );
        dbContextTwo = null;
        connection.Dispose( );
        connection = null;
    }

Stack traces

The stack trace is as follows:

System.NotSupportedException : Collection was of a fixed size.
   at System.SZArrayHelper.Remove[T](T value)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.ClrICollectionAccessor`3.Remove(Object entity, Object value)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.RemoveFromCollection(INavigationBase navigationBase, InternalEntityEntry value)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.RemoveFromCollection(InternalEntityEntry entry, INavigationBase navigation, InternalEntityEntry value)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.KeyPropertyChanged(InternalEntityEntry entry, IProperty property, IEnumerable`1 containingPrincipalKeys, IEnumerable`1 containingForeignKeys, Object oldValue, Object newValue)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.KeyPropertyChanged(InternalEntityEntry entry, IProperty property, IEnumerable`1 keys, IEnumerable`1 foreignKeys, Object oldValue, Object newValue)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectKeyChange(InternalEntityEntry entry, IProperty property)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.PropertyChanged(InternalEntityEntry entry, IPropertyBase propertyBase, Boolean setModified)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.PropertyChanged(InternalEntityEntry entry, IPropertyBase property, Boolean setModified)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetProperty(IPropertyBase propertyBase, Object value, Boolean isMaterialization, Boolean setModified, Boolean isCascadeDelete, CurrentValueType valueType)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetTemporaryValue(IProperty property, Object value, Boolean setModified)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.PropagateValue(InternalEntityEntry principalEntry, IProperty principalProperty, IProperty dependentProperty, Boolean isMaterialization, Boolean setModified)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.SetForeignKeyProperties(InternalEntityEntry dependentEntry, InternalEntityEntry principalEntry, IForeignKey foreignKey, Boolean setModified, Boolean fromQuery)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.InitialFixup(InternalEntityEntry entry, Boolean fromQuery)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.NavigationFixer.StateChanged(InternalEntityEntry entry, EntityState oldState, Boolean fromQuery)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.StateChanged(InternalEntityEntry entry, EntityState oldState, Boolean fromQuery)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.FireStateChanged(EntityState oldState)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState entityState, Boolean acceptChanges, Boolean modifyProperties, Nullable`1 forceStateWhenUnknownKey)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityGraphAttacher.PaintAction(EntityEntryGraphNode`1 node)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityEntryGraphIterator.TraverseGraph[TState](EntityEntryGraphNode`1 node, Func`2 handleNode)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.EntityGraphAttacher.AttachGraph(InternalEntityEntry rootEntry, EntityState targetState, EntityState storeGeneratedWithKeySetTargetState, Boolean forceStateWhenUnknownKey)
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.SetEntityState(InternalEntityEntry entry, EntityState entityState)
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.Add(TEntity entity)
   at CentralTests.Data.Tests.Test2() in C:\Users\Richard\source\repos\MankiRebuild\CentralTests.Data\UnitTest1.cs:line 78



-----

One or more child tests had errors
  Exception doesn't have a stacktrace

Resulting Database

The DataDictionary<>Id column should not exist.

Provider and Version Information

Provider: Sqlite.
Frameworks: NetStandard2.1 for the main project, .Net 6 for the unit tests.
Operating system: Windows 11.
IDE: Jetbrains Rider.
See project files for more details.

[AndriySvyryd]

This should fix the issue:

modelBuilder.Entity<DataDictionary<string, DataObject>>()
    .Ignore(dd => dd.Keys)
    .Ignore(dd => dd.Values);

[rcbellamy]

@AndriySvyryd Could you please point me to the Microsoft-authored documentation that explains why that line is necessary. A few lines of code that one could hardly be expected to guess at the need for certainly does not "fix the issue" at all. For EF Core to be usable, developers have to understand why they do what they do, not just mindlessly copy and paste a "fix" from Microsoft. Without documentation that would enable a developer to figure that out without calling on Microsoft for help, I can hardly imagine what kind of enterprise customer sees any value in paying for Microsoft to develop EF Core at all.

While I do appreciate you providing what appears to me to be a temporarily work-around to something that is not actually behaving the way one would desire or expect, it is absolutely absurd for Microsoft to call this issue "fix[ed]" and conclude that "no[ ]further[ ]action" is necessary on its part. If there is some reason that I am not understanding that this is actually desirable behavior, then please at least point me to the documentation that will fully explain the behavior and the reason that it is considered desirable.

[ajcvickers]

@rcbellamy You are mapping, for example, DataDictionary<string, DataObject> as an entity type. This is, in of itself, a very unusual thing to try to do, since such a type doesn't typically represent a entity type in the application domain. To learn about entity types, I would suggest starting with the Modeling documentation and moving on to Entity Types. This should give a reasonable overview of what entity types represent in EF Core.

If you still want to use DataDictionary<string, DataObject> as an entity type, then it should likely be mapped as a shared-type entity type, since this will allow use of the same CLR type for multiple domain types. This is quite advanced, and is not the way EF is normally used.

In your code, ignoring the Keys and Values properties is needed because, by convention, EF model building configures ICollection<T> properties as navigations to other entity types, as described in the Relationships documentation. EF needs to be configured to not do this, since in this case these properties are not navigations. However, the real issue here, as outlined above, is that using DataDictionary<string, DataObject> as an entity type is probably not a good match for the way EF models a domain.

If you want a book on EF Core, then I would recommend Entity Framework Core in Action by @JonPSmith.

[rcbellamy]

@ajcvickers I appreciate you pointing out what documentation is available, and what book there is to suggest. I have read the first almost 400 pages of EF Core documentation from Microsoft (up to "Migrations Overview[;]" I see that links here work better than they do in the MS-supplied pdf), and after 250 or so pages on the history of EF Core, the portion that I have read did already include the pages to which you have referred me. I note that there is not a single reference to the ICollection interface anywhere in the text of the Relationships page for which you provided a link--not a single mention of it in any discussion. It does show up in code samples, but I see no paragraph stating the convention rule to which you are referring, and what discussion there is combined with making inferences from code is, well, not what I would call "reasonable." As for the notion that the modeling and entity types pages provide a "reasonable overview[,]" it is helpful for me to know what answer Microsoft has, but we shall have to agree to disagree as to what constitutes a "reasonable overview[.]" Obviously, what matters is the opinion of Microsoft's large enterprise customers.

The explanation that you provide in the third paragraph of your response is helpful, and goes beyond that which is explained in the documentation. A mention of a "collection navigation property" does not explain when a collection is considered a collection navigation property. As "unusual" as this use might seem to you, I broke the adequacy of EF Core's documentation with the very first thing that I thought of trying in EF Core. I can hardly imagine professional developers relying on a system that can so quickly put up a roadblock that requires clarification from Microsoft that takes a week to obtain, but I suppose that Microsoft's important customers have a number that they can call to get a much faster and much more thorough explanation.

You implicitly suggest that EF Core might not be the right tool for what I am trying to do. I did consider reading the Ado.Net documentation and using that directly, but I stopped when it became apparent that that documentation (or at least significant portions thereof) is still unaware that the .Net Framework has been replaced by .Net Core. Of course, the 250 pages on the history of EF Core is necessary reading up-front because the remainder of the documentation includes plenty of information that is no longer correct.

As for the "in Action" series, that entire book series needs to be taken out back and shot, but that's not published by Microsoft.