Unable to use camel case in JSON when C# types use pascal case

[Liversage]

One of the shortcomings of the v2 SDK was that when using a custom JSON serializer to convert the pascal cased C# properties to camel case JSON then the LINQ provider would stop working. I raised an issue in the v2 repository about this two years ago.

It seems that this is still a problem in the v3 SDK. Let me walk you through how I tested this. A custom serializer is needed:

class CamelCaseCosmosSerializer : CosmosSerializer
{
    private static readonly Encoding Encoding = new UTF8Encoding(false, true);
    private static readonly JsonSerializer Serializer = new JsonSerializer()
    {
        ContractResolver = new CamelCasePropertyNamesContractResolver()
    };

    public override T FromStream<T>(Stream stream)
    {
        using (stream)
        {
            if (typeof(Stream).IsAssignableFrom(typeof(T)))
                return (T)(object)(stream);

            using (StreamReader streamReader = new StreamReader(stream))
            using (JsonTextReader jsonTextReader = new JsonTextReader(streamReader))
                return Serializer.Deserialize<T>(jsonTextReader);
        }
    }

    public override Stream ToStream<T>(T input)
    {
        var stream = new MemoryStream();

        using (StreamWriter streamWriter = new StreamWriter(stream, encoding: Encoding, bufferSize: 1024, leaveOpen: true))
        using (JsonWriter writer = new JsonTextWriter(streamWriter))
        {
            writer.Formatting = Newtonsoft.Json.Formatting.None;
            Serializer.Serialize(writer, input);
            writer.Flush();
            streamWriter.Flush();
        }

        stream.Position = 0;
        return stream;
    }
}

The document type (notice that the Id property is named using pascal case):

class Document
{
    public string Id { get; set; }
    public string Foo { get; set; }
}

Code to create a document (using the emulator):

var client = new CosmosClientBuilder("AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==")
    .WithCustomSerializer(new CamelCaseCosmosSerializer())
    .Build();
var database = (await client.CreateDatabaseIfNotExistsAsync("Test")).Database;
var container = (await database.CreateContainerIfNotExistsAsync("Test", "/id")).Container;
var id = new Random().Next(1_000_000);
await container.CreateItemAsync(new Document { Id = id.ToString(), Foo = "Bar" });

This works because the custom serializer emits JSON with a lower case id property.

Unfortunately, creating a LINQ query doesn't take this into account:

var queryable = container.GetItemLinqQueryable<Document>().Where(d => d.Foo == "Bar");
Console.WriteLine(queryable.ToString());

Printing queryable results in the following:

{"query":"SELECT VALUE root FROM root WHERE (root[\"Foo\"] = \"Bar\") "}

Notice how the generated query uses pascal cased Foo and not camel cased foo as the property name.

It seems that the LINQ provider in the new SDK (like the old) assumes that .NET property names are mapped to JSON property names without any case conversion making it impossible to use the LINQ provider while at the same time using a JSON serializer that conventionally maps C# pascal case to JSON camel case. I can see that this problem is not easy to solve generally so I would at least like to know if this is a scenario you intend to support in the future.

Interestingly, the query works if I add [JsonProperty("foo")] to the Foo property so there must be some attempt in the LINQ provider to guess how property names are mapped when converting to JSON.

SDK version: 3.0.0, OS version: Windows

[simplynaveen20]

Thanks for the detail description. Based on above basically we are seeing that JsonProperty works but not the custom serializer in LINQ provider. Will have a look in V3 and get back to you .

[simplynaveen20]

As per discussion with @kirankumarkolli , ETA for this feature is next month

[kirankumarkolli]

#716 Adding camel case serialization on LINQ query generation
Above PR address it. It will get released part of 3.2.0

[buyckkdelaware]

#716 Adding camel case serialization on LINQ query generation
Above PR address it. It will get released part of 3.2.0

Is it possible to create a new preview version with this fix included? It's not included in the current 3.2.0-preview version on NuGet. If not, any release data for 3.2.0? Thank you.

[MikesGlitch]

@buyckkdelaware It's released now in Preview 2. Works for me! Thanks guys, this came just in time 🥇

[papadi]

So I understand that this issue was never resolved. The PR mentioned above solves the issue with serialization options of the default serializer are modified. However, @Liversage is using a custom serializer and in that case, setting serialization options for the default serializer isn't possible.

[Liversage]

@papadi I believe that I'm using a custom serializer in my code because that was the only way to change the casing of the JSON at the time of writing. With the fix I no longer need a custom serializer.

Now, if you are using a custom serializer you might run into issues with LINQ as you remark. I have since raising this issue concluded that it's just too hard to work with libraries like this that want to serialize JSON and at the same time try to control how the serialization is done. Alternatively, you can add a mapping layer where you transform your domain models that might require tweaks to the serializer into more generic models that serialize without issues. Adding this mapping layer is of course non-trivial especially if you also want to map "domain LINQ queries" to "mapped LINQ queries" but it's possible.

[papadi]

Yes, I still do need a custom serializer for some other reason. The workaround for the moment is to decorate properties with JsonProperty attribute and define the camel case name there.

I guess another workaround could be to create another instance of CosmosClient using camel serialization setting without the custom serializer, create the query with that one using LINQ and then pass the query string to the original container. :P

I see you have opened another ticket to fix this, but I guess it hasn't attracted a lot of attention yet.