ArrayPool does not resize based on user request

[FrediKats]

ArrayPool

Description

ArrayPool (TlsOverPerCoreLockedStacksArrayPool) does not resize based on user request. TlsOverPerCoreLockedStacksArrayPool has constant value for arrays buckets. If those number of arrays is not enough then we get GC.AllocateUninitializedArray call:

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Buffers/TlsOverPerCoreLockedStacksArrayPool.cs#L24

This approach is good enough if we can take and return array with low delay but can lead to issues in case when we need many arrays for a long time. I compare ArrayPool<T>.Shared and ArrayPool<T>.Create. We ran into a problem while checking solution with genetic algorithm. In our case it needs to allocate about 3000 arrays (with constant size) for one iteration then we return it back to array pool and want to reuse in next iterations.

Repository with reproduce is available here - https://github.com/FrediKats/ArrayPoolAllocationReproduce. You can just run this project as-is and check allocations.

• .Benchmark contains benchmark source code
• .MemoryTest contains code that was run under dotMemory

Configuration

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
AMD Ryzen 7 5800H with Radeon Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100-preview.4.22252.9
  [Host]     : .NET 6.0.5 (6.0.522.21309), X64 RyuJIT
  DefaultJob : .NET 6.0.5 (6.0.522.21309), X64 RyuJIT

Method	UseShared	Mean	Error	StdDev	Gen 0	Gen 1	Allocated
Read	False	274.1 ms	4.16 ms	5.96 ms	-	-	5 MB
Read	True	292.4 ms	4.52 ms	3.78 ms	4000.0000	2000.0000	39 MB

dotMemory traces

DotMemory trace with changing allocation mode:

Allocated objects while we using ArrayPool<Vector2>.Shared:

Allocated objects while we using pool form ArrayPool<Vector2>.Create:

Analysis

Same issue was mentioned in #52098 (comment).

[ghost]

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

Issue Details

---

ArrayPool

Description

ArrayPool (TlsOverPerCoreLockedStacksArrayPool) does not resize based on user request. TlsOverPerCoreLockedStacksArrayPool has constant value for arrays buckets. If those number of arrays is not enough then we get GC.AllocateUninitializedArray call:

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Buffers/TlsOverPerCoreLockedStacksArrayPool.cs#L24

This approach is good enough if we can take and return array with low delay but can lead to issues in case when we need many arrays for a long time. I compare ArrayPool<T>.Shared and ArrayPool<T>.Create. We ran into a problem while checking solution with genetic algorithm. In our case it needs to allocate about 3000 arrays (with constant size) for one iteration then we return it back to array pool and want to reuse in next iterations.

Repository with reproduce is available here - https://github.com/FrediKats/ArrayPoolAllocationReproduce. You can just run this project as-is and check allocations.

• .Benchmark contains benchmark source code
• .MemoryTest contains code that was run under dotMemory

Configuration

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
AMD Ryzen 7 5800H with Radeon Graphics, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100-preview.4.22252.9
  [Host]     : .NET 6.0.5 (6.0.522.21309), X64 RyuJIT
  DefaultJob : .NET 6.0.5 (6.0.522.21309), X64 RyuJIT

Method	UseShared	Mean	Error	StdDev	Gen 0	Gen 1	Allocated
Read	False	274.1 ms	4.16 ms	5.96 ms	-	-	5 MB
Read	True	292.4 ms	4.52 ms	3.78 ms	4000.0000	2000.0000	39 MB

dotMemory traces

DotMemory trace with changing allocation mode:

Allocated objects while we using ArrayPool<Vector2>.Shared:

Allocated objects while we using pool form ArrayPool<Vector2>.Create:

Analysis

Same issue was mentioned in #52098 (comment).

Author:	FrediKats
Assignees:	-
Labels:	area-System.Buffers, tenet-performance, untriaged
Milestone:	-

[En3Tho]

This is specifically why custom Array pool creation api was made, isn't it? Shared array pool is used across whole runtime and is used specifically with quick take-give back in mind.

I doubt that tuning Shared for such a specific use case makes sense.

[AaronRobinsonMSFT]

/cc @Maoni0 in relation to #52098.