EFCore 8 RC cannot map (dynamic / object) in complex type

[xevilmaxx]

Hi, was trying out RC1 release.

public class DynamicKey_Table
{
    public long Id { get; set; }
    public List<DynamicKey> Keys { get; set; }
}

public class DynamicKey
{
    public string Key { get; set; }
    public dynamic Value { get; set; }
}

So instead of OwnsOne/OwnsMany i started to use ComplexTypes.
Here i needed to define also Keys as complex property becouse it cannot go without IsRequired(), but i assume this issue is already known and probably will be fixed in future releases

//inside OnConfiguring method

modelBuilder.Entity<DynamicKey_Table>(e =>
{
    
    e.Property(x => x.Id).UseHiLo(nameof(DynamicKey_Table));
    
    e.ComplexProperty(x => x.Keys).IsRequired();

    e.ComplexProperty(
        x => x.Keys,
        y => y.ComplexProperty(z => z.Value).IsRequired()
        );

});

It gives me following error:

Unable to create a 'DbContext' of type 'PostgresContext'. The exception 'Complex type 'DynamicKey_Table.Data#DynamicKey.Value#object' has no properties defines. Configure at least one property or don't include this type in the model.' was thrown while attempting to create an instance. For the different patterns supported at design time, see https://go.microsoft.com/fwlink/?linkid=851728

Basically i cant map object/dynamic property type in any way.

Other frameworks like Marten / Npgsql can handle it, sometimes they use syntax similar to this:

    {
      "Key": "RemoteGRPC",
      "Value": {
        "Ip": "127.0.0.1",
        "Port": 1111,
        "$type": "BeaconLib.DTO.DiscoveredData, BeaconLib",
        "GrpcEndpoint": ""
      },
      "Description": "Populate only if autodiscovery has problems"
    }

Where $type is the attribute added by the framework to memorize data type.
In this case -> BeaconLib is a project of type library, while the rest is the namespace in that project.

I don't mind to be able to do queries on dynamic type, but simply would like to be able to store it and retrieve in memory after deserialization from DB, than i will do my operations to handle it in the way i need.

[roji]

It sounds like you're looking for something like weakly-typed JSON mapping. We have #28871 tracking doing this via JsonDocument/JsonElement, #29825 for doing it via a dictionary, and this issue requests for doing this via dynamic.

Unfortunately dynamic is not supported in expression trees; even if expression trees are evolved to support new C# language syntax, it's quite unlikely that dynamic will get support.

[xevilmaxx]

Correct me if i'm wrong:

• We use Expression Trees to basically parse linq syntax and transform it into destination language (sql or pseudo sql in this case).

i wonder if we can treat dynamic as a "black box"?

in a way that:

• framework can serialize it properly and memorize on db (inside json struct)
• being able to retrieve it and deserialize back into dynamic object (with original type)

assumptions:

• you cannot perform queries on dynamic property
• but you can do this on parent type
  • var keyObj = db.DynamicKey_Table.First().Keys.First();
  • or
  • var keyObj = db.DynamicKey_Table.Keys.Where(x => x.Key == "RemoteGRPC").First();
  • in this case i will have DynamicKey object and i know that inside it has property Value, which is "something"
  • than i can decide what to do with this on runtime

In this way all logic of Expression Tree is fully bypassed i think.

Otherwise i can obviously treat it like a string myself and perform some logics on serialization/deserialization.
The only drawback is that if Value is of type int, i will be forced to memorize it like: {"Value": "1"} instead of {"Value": 1}

While your framework probably already has the ability to understand the data type and serialize it properly.

[roji]

you cannot perform queries on dynamic property
but you can do this on parent type

What does a "dynamic table" map to in the database? A dynamic property could map to a JSON column in the database (but that's not possible because of the compiler limitation on dynamic in expression trees), but there's no such thing as a "dynamic table" in relational databases. In theory, this may make sense in non-relational document databases (e.g. Cosmos, MongoDB), but I don't think that's what you're asking about.

Even if this somehow made sense for relational databases, this means you once again can't reference the dynamic table (db.DynamicKey_Table) within a LINQ query; that would mean that subqueries wouldn't be supported (i.e. db.SomeTable.Where(t => db.DynamicKey_Table.First()...)). So I don't think this makes sense.

[xevilmaxx]

Dynamic_Table will map to DB in structure similar to this:

Idk, what you mean about compiler limitation.

Here is a working spinnet to understand if property is a dynamic type:

using System;
using System.Reflection;
using System.Runtime.CompilerServices;
					
public class Program
{
	
	public class Test0
	{
		public string Tmp { get; set; }
	}
	
	public class Test
        {
                public string Name { get; set; }
                public dynamic Value { get; set; }
		public Test0 Obj { get; set; }
        }
	
	public static void Main()
	{
            var test = new Test();
            var properties = test.GetType().GetProperties();
            foreach (var property in properties)
            {
                bool isDynamic = property.GetCustomAttributes(typeof(DynamicAttribute), true).Length > 0;
                Console.WriteLine(property.Name + (isDynamic ? " is dynamic" : " is not dynamic"));
            }
	}
	
}

Output:

Name is not dynamic
Value is dynamic
Obj is not dynamic

Can't it be simply treated like ComplexObject (or some subtype) and stored to Db with type, and once fetched back deserialized to that type?

Obviously you won't be able to do linq query on "dynamic" property, but you know that if you query a known property type, it will also bring back dynamic with it.

[roji]

Idk, what you mean about compiler limitation.
[...]
Obviously you won't be able to do linq query on "dynamic" property, but you know that if you query a known property type, it will also bring back dynamic with it.

Your code sample doesn't contain a LINQ query (expression trees). LINQ is the way that EF supports querying; if you can't e.g. filter by a property - because the compiler doesn't support dynamic in expression trees - then that's a complete non-starter which would make this feature useless.

Dynamic_Table will map to DB in structure similar to this:

If what you want is a table with an ID and a JSON field, then you can model it that way; we generally consider it out of scope for EF to do this kind of map. There's also very little value in EF doing it for you when you can just do it yourself.

[xevilmaxx]

Ok,

Suppose dynamic is of real type is primitive (int/bool/...).

Can you suggest any way in order to obtain following json on db:
{"Value": 9999}
instead of
{"Value": "9999"}

or

{"Value": true}
instead of
{"Value": "true"}

do you think this may do the trick? :

public class DynamicValueConverter : ValueConversion<dynamic, string>
{
    public DynamicValueConverter() : base(
        v => JsonConvert.SerializeObject(v, new JsonSerializerOptions
        {
            WriteIndented = false,
            WriteNumbersWithQuotes = false,
            WriteBooleansWithQuotes = false
        }),
        s => JsonConvert.DeserializeObject<dynamic>(s))
    {
    }
}

or

public class DynamicValueConverter : ValueConversion<dynamic, JsonDocument>
{
    public DynamicValueConverter() : base(
        v => JsonDocument.Parse(JsonSerializer.Serialize(v)),
        s => JsonSerializer.Deserialize<dynamic>(s.RootElement.GetRawText()))
    {
    }
}

Probably JsonDocument sould be treated as partial json, so it's much more appropriate

public class MyEntity
{
    public int Id { get; set; }

    [ValueConverter(typeof(DynamicValueConverter))]
    public dynamic DynamicProperty { get; set; }
}

[roji]

You'll have to figure out how to work with dynamic and convert it to the JSON document you want - I'm not extremely familiar with it.

[xevilmaxx]

Ok, i finally was able to create the migration.
I saw that complex type properties are translated into columns of a table.

Probably initially i misunderstood it's purpose.
It's now obvious that my request makes no sense in this scenario.

So, will wait to see how things will evolve when there will be the feature to map complex type into single json column.
Hope that in that case eventually also dynamic/object will be allowed.

For now will try to play with EfCore Npgsql, which is more json oriented.

[roji]

So, will wait to see how things will evolve when there will be the feature to map complex type into single json column.
Hope that in that case eventually also dynamic/object will be allowed.

We definitely plan allowing the new "complex type" feature to map to JSON columns in EF Core 9.0 - though you can already use owned types to do that today; they are conceptually very similar.

However, there's definitely no plan to allow complex type to work with dynamic/object; the whole point of complex type modeling is that EF is aware of the type and it's structure; that's the opposite of dynamic mapping, where EF knows nothing about the shape and contents of your JSON document.

The Npgsql provider is very similar in this respect: it allows you to map arbitrary user POCOs to a JSON document, but you still need to have a POCO type. The Npgsql provider does allow mapping via JsonDocument/JsonElement - which are indeed fully dynamic - but not via the C# dynamic feature.