Serialization / CosmosElements: Adds LazyCosmosElement Cache Improvements

[bchong95]

Serialization / CosmosElements: Adds LazyCosmosElement Cache Improvements

Description

Improves the cache for LazyCosmosElements. After adding a benchmark LazyCosmosElements it was evident that the cacheing story was not exhaustive. Enumerating would always reparse the buffer for the JsonElement instead of serving from the cache. The new design is just initializing LazyCosmosElements for every element in the array or object. This is fine since the elements are lazy and cheap to create (by the recursive nature of lazy JSON). This all of nothing approach leads to less state management, which turns out to be more performant then the previous conservative approach.

Also for Array and Object enumerators we return a struct now instead of interface to avoid boxing. This makes the enumeration code paths allocation free.

Benchmarks:

Before:

|------------------------------- |------------ |------------------------ |----------- |-------------:|-------------:|-------------:|-------------:|---------:|------:|------:|----------:|
| CreateAndNavigateCosmosElement |       array |                    Text |      Never |     186.1 ns |      4.04 ns |      6.64 ns |     183.2 ns |   0.0975 |     - |     - |     408 B |
| CreateAndNavigateCosmosElement |       array |                    Text |       Once |   5,386.7 ns |    104.81 ns |    102.93 ns |   5,410.2 ns |   1.1902 |     - |     - |    5000 B |
| CreateAndNavigateCosmosElement |       array |                    Text |       Many |   9,549.7 ns |    204.79 ns |    191.56 ns |   9,493.5 ns |   1.2970 |     - |     - |    5432 B |
| CreateAndNavigateCosmosElement |       array |                  Binary |      Never |     160.8 ns |      3.22 ns |      3.17 ns |     160.0 ns |   0.0629 |     - |     - |     264 B |
| CreateAndNavigateCosmosElement |       array |                  Binary |       Once |   3,714.9 ns |     40.81 ns |     36.18 ns |   3,714.4 ns |   1.1826 |     - |     - |    4960 B |
| CreateAndNavigateCosmosElement |       array |                  Binary |       Many |   8,062.9 ns |    167.38 ns |    205.56 ns |   8,034.8 ns |   1.2817 |     - |     - |    5392 B |
| CreateAndNavigateCosmosElement |      double |                    Text |      Never |     621.1 ns |     12.43 ns |     35.07 ns |     610.1 ns |   0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |      double |                    Text |       Once |     736.3 ns |     14.75 ns |     13.08 ns |     735.4 ns |   0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |      double |                    Text |       Many |     885.5 ns |     17.67 ns |     40.94 ns |     875.2 ns |   0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |      double |                  Binary |      Never |     161.5 ns |      2.82 ns |      2.50 ns |     162.1 ns |   0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement |      double |                  Binary |       Once |     245.8 ns |      5.37 ns |      5.28 ns |     244.5 ns |   0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement |      double |                  Binary |       Many |     358.0 ns |      6.70 ns |      6.88 ns |     356.2 ns |   0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement | long string |                    Text |      Never |   1,665.2 ns |     29.77 ns |     27.85 ns |   1,660.0 ns |   0.1087 |     - |     - |     456 B |
| CreateAndNavigateCosmosElement | long string |                    Text |       Once |   2,080.1 ns |     39.18 ns |     36.65 ns |   2,086.6 ns |   0.3242 |     - |     - |    1368 B |
| CreateAndNavigateCosmosElement | long string |                    Text |       Many |   2,105.3 ns |     34.39 ns |     32.17 ns |   2,102.8 ns |   0.3242 |     - |     - |    1368 B |
| CreateAndNavigateCosmosElement | long string |                  Binary |      Never |     161.0 ns |      3.23 ns |      4.31 ns |     160.0 ns |   0.0629 |     - |     - |     264 B |
| CreateAndNavigateCosmosElement | long string |                  Binary |       Once |   1,381.4 ns |     17.76 ns |     14.83 ns |   1,380.0 ns |   0.2804 |     - |     - |    1176 B |
| CreateAndNavigateCosmosElement | long string |                  Binary |       Many |   1,466.6 ns |     20.55 ns |     19.23 ns |   1,462.4 ns |   0.2804 |     - |     - |    1176 B |
| CreateAndNavigateCosmosElement |      object |                    Text |      Never |     315.1 ns |      5.63 ns |      5.27 ns |     313.4 ns |   0.2637 |     - |     - |    1104 B |
| CreateAndNavigateCosmosElement |      object |                    Text |       Once |  45,597.9 ns |    624.23 ns |    583.91 ns |  45,734.8 ns |   9.4604 |     - |     - |   39728 B |
| CreateAndNavigateCosmosElement |      object |                    Text |       Many | 314,062.6 ns |  4,329.06 ns |  4,049.40 ns | 313,535.7 ns |  77.6367 |     - |     - |  324712 B |
| CreateAndNavigateCosmosElement |      object |                  Binary |      Never |     288.7 ns |      5.37 ns |      4.48 ns |     287.3 ns |   0.2294 |     - |     - |     960 B |
| CreateAndNavigateCosmosElement |      object |                  Binary |       Once |  89,341.4 ns |  1,778.45 ns |  2,374.18 ns |  89,078.8 ns |  23.9258 |     - |     - |  100360 B |
| CreateAndNavigateCosmosElement |      object |                  Binary |       Many | 697,750.9 ns | 13,902.99 ns | 22,843.01 ns | 695,548.6 ns | 209.9609 |     - |     - |  880768 B |

After:

|                         Method | payload | jsonSerializationFormat | accessMode |        Mean |     Error |    StdDev |      Median |  Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------------------- |-------- |------------------------ |----------- |------------:|----------:|----------:|------------:|-------:|------:|------:|----------:|
| CreateAndNavigateCosmosElement |   array |                    Text |      Never |    193.8 ns |   5.39 ns |   8.07 ns |    190.3 ns | 0.1051 |     - |     - |     440 B |
| CreateAndNavigateCosmosElement |   array |                    Text |       Once |  4,927.2 ns |  78.67 ns |  69.74 ns |  4,928.7 ns | 0.7935 |     - |     - |    3336 B |
| CreateAndNavigateCosmosElement |   array |                    Text |       Many | 10,528.4 ns | 113.33 ns | 100.46 ns | 10,506.0 ns | 0.7935 |     - |     - |    3336 B |
| CreateAndNavigateCosmosElement |   array |                  Binary |      Never |    163.1 ns |   1.00 ns |   0.93 ns |    163.4 ns | 0.0706 |     - |     - |     296 B |
| CreateAndNavigateCosmosElement |   array |                  Binary |       Once |  3,368.9 ns |  57.71 ns |  51.16 ns |  3,355.7 ns | 0.7858 |     - |     - |    3296 B |
| CreateAndNavigateCosmosElement |   array |                  Binary |       Many |  8,799.1 ns | 131.52 ns | 109.83 ns |  8,794.9 ns | 0.7782 |     - |     - |    3296 B |
| CreateAndNavigateCosmosElement |  double |                    Text |      Never |    599.0 ns |  14.22 ns |  13.30 ns |    593.3 ns | 0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |  double |                    Text |       Once |    728.3 ns |  10.22 ns |   9.56 ns |    729.4 ns | 0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |  double |                    Text |       Many |    970.9 ns |  19.40 ns |  17.20 ns |    967.3 ns | 0.1183 |     - |     - |     496 B |
| CreateAndNavigateCosmosElement |  double |                  Binary |      Never |    158.8 ns |   2.96 ns |   2.77 ns |    158.8 ns | 0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement |  double |                  Binary |       Once |    260.2 ns |   5.37 ns |   5.02 ns |    260.0 ns | 0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement |  double |                  Binary |       Many |    481.5 ns |   6.05 ns |   5.66 ns |    480.1 ns | 0.0725 |     - |     - |     304 B |
| CreateAndNavigateCosmosElement |  object |                    Text |      Never |    186.6 ns |   3.74 ns |   7.21 ns |    183.6 ns | 0.0975 |     - |     - |     408 B |
| CreateAndNavigateCosmosElement |  object |                    Text |       Once | 15,670.8 ns | 289.71 ns | 256.82 ns | 15,602.5 ns | 1.4648 |     - |     - |    6224 B |
| CreateAndNavigateCosmosElement |  object |                    Text |       Many | 47,673.3 ns | 906.89 ns | 848.31 ns | 47,446.5 ns | 1.4648 |     - |     - |    6224 B |
| CreateAndNavigateCosmosElement |  object |                  Binary |      Never |    159.7 ns |   2.19 ns |   2.05 ns |    159.9 ns | 0.0629 |     - |     - |     264 B |
| CreateAndNavigateCosmosElement |  object |                  Binary |       Once | 12,350.7 ns | 254.57 ns | 584.92 ns | 12,086.6 ns | 1.4648 |     - |     - |    6184 B |
| CreateAndNavigateCosmosElement |  object |                  Binary |       Many | 43,907.8 ns | 567.91 ns | 503.44 ns | 43,908.0 ns | 1.4648 |     - |     - |    6184 B |
| CreateAndNavigateCosmosElement |  string |                    Text |      Never |  1,830.3 ns |  24.70 ns |  21.90 ns |  1,831.6 ns | 0.1087 |     - |     - |     456 B |
| CreateAndNavigateCosmosElement |  string |                    Text |       Once |  1,953.0 ns |  26.04 ns |  24.36 ns |  1,956.9 ns | 0.3242 |     - |     - |    1368 B |
| CreateAndNavigateCosmosElement |  string |                    Text |       Many |  2,204.6 ns |  30.17 ns |  26.74 ns |  2,202.8 ns | 0.3242 |     - |     - |    1368 B |
| CreateAndNavigateCosmosElement |  string |                  Binary |      Never |    161.8 ns |   3.32 ns |   3.11 ns |    162.0 ns | 0.0629 |     - |     - |     264 B |
| CreateAndNavigateCosmosElement |  string |                  Binary |       Once |  1,418.2 ns |  17.42 ns |  14.54 ns |  1,412.6 ns | 0.2804 |     - |     - |    1176 B |
| CreateAndNavigateCosmosElement |  string |                  Binary |       Many |  1,580.7 ns |  19.41 ns |  18.16 ns |  1,582.4 ns | 0.2804 |     - |     - |    1176 B |

For arrays we are looking at about 33% less memory utilization for the once and many scenario

For objects we are looking at 1/4th the memory for never, 1/100th for once, and 1/400th for many.

Unit tests were also added to cover these optimizations.

[john-pao]

Overall, the caching implementation is about what I would expect. A few concerns about defining specific implementations on the public surface area for CosmosArray/CosmosObject, as it establishes a precedent that may be detrimental to us going forward.

[ghost]

Closing due to in-activity, pease feel free to re-open.