Using ODataUtf8JsonWriter leads to increased memory usage and latency when writing properties with large string or byte[] values

[habbes]

We have customers with scenarios that write payloads with really large string values. For example, string property values with 18 million characters. This causes problems when using ODataUtf8JsonWriter because Utf8JsonWriter.WriteStringValue() will write that entire string to the buffer all at once. Since this exceeds the buffer size, it will allocate a buffer large enough to fit the entire string in memory. If the string needs escaping, it will also allocate a buffer large enough to store the escaped version of the string. This causes issues with large-object heap (LOH) allocations, increase in latency due to increased GC pressure, etc.

This is a known issue of Utf8JsonWriter. At the time of this writing there's a TODO item in the Utf8JsonWriter source

that references the following issue: dotnet/runtime#29293 from 2019. That issue proposes writing large strings in smaller 4KB chunks instead of all at once. That issue seems to have been closed due to inactivity. But there's an other open issue with an active discussion thread and design proposals dotnet/runtime#67337 that seems to have caught some momentum.

Meanwhile, we could solve this in our library by detecting large strings based on some threshold, write to the buffer in chunks and flush it to the stream intermittently before getting to the point where the buffer needs to be resized. Ironically, our default JsonWriter gets this for free by using the TextWriter.

The drawback of doing this ourselves (at least until Utf8JsonWriter supports it), is that we'll have to handle string escaping, and write the separator manually. Luckily, we already dealt with the separator issue when working on raw values: #2527

NOTE: This issue also applies to writing byte[] arrays, and possibly to raw values. I don't think property names and other primitive values present a concern because they tend to be short.

Assemblies affected

Microsoft.OData.Core 7