LINQ-constructed document queries ignore Json annotations based on type constraint

[fabianmeyer]

Describe the bug
When using LINQ to create queries in a generic function with a constraint on the document's type parameter, the query generator ignores annotations on the concrete type. Specifically, the generator ignores Json serialization annotations on the properties, leading to incorrect queries.

To Reproduce

using System;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Microsoft.Azure.Cosmos.Fluent;
using Newtonsoft.Json;

namespace ConsoleApp
{
    interface ITest
    {
        string Id { get; set; }
        
        string Name { get; set; }
    }

    class Test : ITest
    {
        [JsonProperty("id")] 
        public string Id { get; set; }
        
        [JsonProperty("myweirdnameproperty")] 
        public string Name { get; set; }
    }
    
    class Program
    {
        static async Task Main(string[] args)
        {
            
            var client = new CosmosClientBuilder("AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==").Build();
            
            
            var databaseResp = await client.GetDatabase("testdb").ReadAsync();
            var database = databaseResp.StatusCode == HttpStatusCode.OK
                ? databaseResp.Database
                : (await client.CreateDatabaseAsync("testdb")).Database;
            
            var containerResp = await database.GetContainer("testcontainer").ReadContainerAsync();
            var container = containerResp.StatusCode == HttpStatusCode.OK
                ? containerResp.Container
                : (await database.CreateContainerAsync(new ContainerProperties("testcontainer", "/id"))).Container;
            
            
            InterfaceConstraint<Test>(container);
            ClassConstraint<Test>(container);
        }

        private static void InterfaceConstraint<T>(Container container)
        where T :  ITest
        {
            Console.WriteLine($"InterfaceConstraint Query for T = \"{typeof(T)}\": {container.GetItemLinqQueryable<T>().Select(doc => doc.Name).ToSqlQueryText()}");
        }
        
        private static void ClassConstraint<T>(Container container)
            where T :  Test
        {
            Console.WriteLine($"ClassConstraint Query for T = \"{typeof(T)}\": {container.GetItemLinqQueryable<T>().Select(doc => doc.Name).ToSqlQueryText()}");
        }
    }    
}

Expected behavior

Query generator should consider annotation on Test class. Expected output:

InterfaceConstraint Query for T = "ConsoleApp.Test": SELECT VALUE root["myweirdnameproperty"] FROM root
ClassConstraint Query for T = "ConsoleApp.Test": SELECT VALUE root["myweirdnameproperty"] FROM root

Actual behavior

Query generator ignores annotation on Test class. Actual output:

InterfaceConstraint Query for T = "ConsoleApp.Test": SELECT VALUE root["Name"] FROM root
ClassConstraint Query for T = "ConsoleApp.Test": SELECT VALUE root["myweirdnameproperty"] FROM root

Environment summary
SDK Version: Microsoft.Azure.WebJobs.Extensions.CosmosDB 3.0.3, .NET Core 2.2
OS Version: Windows 10,

[j82w]

This is an interesting problem. In v3 we allow a custom serializer. A lot of people have shown interest in using .NET core 3.0 system.text.json serializer because of the performance increase it has over Json.NET. Only option I see is if we added some support for a custom serializer for the property names.

@simplynaveen20 If we just assume the user is always using the default Json.NET implementation could we add support for Json.NET attribute tags?

[fabianmeyer]

What seems weird to me is that the query builder behaves differently depending on the constraint on T. When the concrete type is used as constraint, the annotations are considered correctly. When the interface is used, the annotations are not considered. Even though T is the same concrete type in both cases and the annotations are present on T in both functions per reflection.

[j82w]

The reason this is happening is the constraint on T causes everything to be down cast to ITest as the type. Based on the code right now we don't check for derived types. You can see the code here.

[simplynaveen20]

Will have a look to this POST GA

[lommez]

Any updates on this issue?
I need so much to use the JsonIgnore attribute

[holietenk]

Any updates?

The following changes in ExpressionToSQL.cs work for me:

PropertyInfo targetProperty = inputExpression.Expression.Type.GetProperty(inputExpression.Member.Name);

string memberName = targetProperty.GetMemberName(context?.serializationOptions);

Of course, this is just for demo purpose only ... :)

[j82w]

@bchong95 can you take a look?

[holietenk]

any updates? @bchong95 @j82w
do you have plans to fix this?

[thomaslevesque]

Workaround, until this is fixed: apply the JsonProperty attributes on the interface properties. It's ugly, but it works.
(in addition to the class, otherwise other things will break...)

[CharlieDigital]

@thomaslevesque Upvote for the simple solution; banging my head against this last night for a few hours!

[AndrewTriesToCode]

Is this something that will be addressed in V3 or should we plan to work around?

[jcocchi]

@leminh98 can you take a look at this?

[Maya-Painter]

Hi all, we've merged what we hope will be a solution to this issue by allowing the custom serializer to be used for LINQ translations. This also unblocks having System.Text.Json attributes applied in LINQ queries.

We plan to release this as part of the next preview package, but I also wanted to take this time to confirm these changes meet your needs before finalizing them as part of our public API. If you try the changes out, please follow up in the thread if this update addresses your concerns. If it doesn't, please let us know why.

To use your own custom serializer for LINQ translations you must implement CosmosLinqSerializer and set it as a custom serializer on the CosmosClient. There is a sample of this for serialization with System.Text.Json in CosmosLinqSerializer.cs.