list of usage examples that illustrate current problems with ArrayPool and what the desired behavior should be

this issue is for folks who want ArrayPool (or other memory pools) to be improved. please explain it with usage examples. a usage example should consist of -

pseudo code snippet to illustrate your scenario
description of current problems
description of your desired behavior, eg, your pool is obviously part of the memory usage in a process, do you want your pool to have an individual threshold that you get notified when it's exceeded and you will do something about it, or do you want to with a default size and leave it up to whenever GC decides to collect.
any other perf implications you want to mention, eg, does your pool only manage buffers that will need to pinned, what kind of objects do you want to have in this pool, things like byte[] only or any type

then we have a discussion what kinds of behavior our pooling should offer and then we can decide if we need support from the GC and what that would be.

Problem 1: ArrayPool does not ever release pooled memory, unless there is a memory pressure.

using System;
using System.Buffers;

DoStuff<int>();
DoStuff<uint>();
DoStuff<long>();
DoStuff<ulong>();
DoStuff<DateTime>();
DoStuff<DateTimeOffset>();

// Let's say the above was done during startup and the rest of the application won't use those ArrayPools again

Console.WriteLine(GC.GetTotalMemory(true));

void DoStuff<T>()
{
   ArrayPool<T>.Shared.Return(ArrayPool<T>.Shared.Rent(1000000));
}

Actual behavior: Array pools are keeping 50MB forever
Expected behavior: The memory is released after it is not used for a while

dotnet/runtime libraries are reluctant to use ArrayPools for anything but byte and char to avoid hitting this issue.

Problem 2: Array pool does not pool enough

using System;
using System.Buffers;

for (int i = 0; i < 1000000; i++)
    Nested(100);

Console.WriteLine(GC.CollectionCount(0));

void Nested(int depth)
{
   if (depth == 0) return;

   var a = ArrayPool<byte>.Shared.Rent(10);
   Nested(depth-1);
   ArrayPool<byte>.Shared.Return(a);
}

Expected: 0 or very few GCs
Actual: Hundreds of GCs

We have to watch for using the array pool too much that would lead to hitting the threasholds. The Array pool should dynamically resize based on the application behavior.

Problem 3: ArrayPool does not work for very large arrays

using System;
using System.Buffers;

for (int i = 0; i < 1000; i++)
   ArrayPool<byte>.Shared.Return(ArrayPool<byte>.Shared.Rent(10_000_000));

Console.WriteLine(GC.CollectionCount(2));

Expected: 0 or very few GCs
Actual: Hundred of Gen2 GCs

This implementation limitation largely exists to reduce severity of Problem 1.

A side-effect is that code tends to use array pool limits for buffer sizes instead of figuring about what the rigth buffer size should actually be. Example from this week: https://github.com/dotnet/runtime/pull/49304/files#diff-7721d7d6b3bf50b13337accfc3a3857f235a3901e99c6f0a9ff4c2c5f84c2784R49

@davidfowl

cc @davidfowl

@jkotas

thanks @jkotas! I will take a look at these hopefully in the next couple of weeks.

Important aspect of Problem 1 (#52098 (comment)) is that ArrayPools are per-type. A 3rd library party may unintentionally retain a bunch of memory by using ArrayPool<SomeLargeUniqueStruct>.Shared (this actually happened in an early alpha of ImageSharp).

Another thing I would like to do is get rid of the Kestrel memory pool since it's at this point a glorified bunch of pooled buffers 4K on the pinned object heap. Today these ArrayPool buffers aren't pinned making them less than ideal for networking scenarios.

@jkotas

Problem 2: Array pool does not pool enough
using System;
using System.Buffers;

for (int i = 0; i < 1000000; i++)
    Nested(100);

Console.WriteLine(GC.CollectionCount(0));

void Nested(int depth)
{
   if (depth == 0) return;

   var a = ArrayPool<byte>.Shared.Rent(10);
   Nested(depth-1);
   ArrayPool<byte>.Shared.Return(a);
}
Expected: 0 or very few GCs
Actual: Hundreds of GCs

We have to watch for using the array pool too much that would lead to hitting the threasholds. The Array pool should dynamically resize based on the application behavior.

@jkotas for this problem I cannot repro what you saw. I printed out a bit more info -

elapsed 28222 ms, 0 gen0 GCs, 0 gen1, 0 gen2, allocated 17,264 bytes

it allocates very little and no GCs were triggered. this is with the current .net 6 build.

I think this issue is about revisiting the current tuning. On the current master, the following code will retain 2GB of memory, without ever freeing it up, using ArrayPool<byte>.Shared only:

for (long size = 1024 * 1024; size <= 1073741824; size *= 2)
{
    byte[] a = ArrayPool<byte>.Shared.Rent((int)size);
    ArrayPool<byte>.Shared.Return(a);
}

While this might seem an artificial scenario, there are use cases which could make it very real. Imagine an application working with very large images (of different sizes) assuming the underlying imaging library relies heavily on ArrayPool<byte>.Shared. The application might be very heavy on imaging operations during a warmup time, but not doing them anymore in the rest of it's execution time (eg. because of caching), so retaining the arrays doesn't make sense anymore. This is essentially what's SixLabors/ImageSharp#1590 is about, with the difference that currently we are using a custom instance of ConfigurableArrayPool<byte> to pool very large arrays.

I think this issue is about revisiting the current tuning.

You asked "What would the runtime/BCL fix to this problem look like?" and I'm answering that question: tune the Trim method.

#58974 is another example of unexpected behavior that comes as a surprise: ArrayPool does not always work well when renting/returning happens on different threads.

The memory pressure is determined through GC.GetGCMemoryInfo().HighMemoryLoadThresholdBytes ( https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Buffers/Utilities.cs#L38-L56 ).

On docker containers that enforce a cgroup memory limit, the value for HighMemoryLoadThresholdBytes is actually greater than the TotalAvailableMemoryBytes. ( #58974 (comment) )

This means when running inside a container with a cgroup limit, the app will always OOM before Utilities.GetMemoryPressure() returns MemoryPressure.High ( because the threshold for MemoryPressure.High exceeds the amount of memory available ). The window for it to return return MemoryPressure.Medium is also much smaller.

Perhaps consider either applying the fix proposed in #58974 (comment) or changing the logic to use TotalAvailableMemoryBytes instead. 👍

The memory pressure is determined through GC.GetGCMemoryInfo().HighMemoryLoadThresholdBytes ( https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Buffers/Utilities.cs#L38-L56 ).

This should also consider load of the managed heap in a container, and that will trigger aggressive trimming in @Bio2Hazzard's example application: https://gist.github.com/Bio2hazard/ee353c1042ee56a97c0d0b3d62c590bc.

I was surprised that ArrayPool<T>.Shared doesn't actually seem to share pooled arrays between threads. I would expect the following code to print "Got uninitialized array" just once, but it prints this message 10 times:

using System.Buffers;

var pool = ArrayPool<int?>.Shared;

// Uncomment next line to get expected behavior
//pool = ArrayPool<int?>.Create(1024 * 1024, 50);

for (int i = 0; i < 10; i++)
{
    var th = new Thread(() =>
    {
        var arr = pool.Rent(1);

        if (arr[0] == null)
        {
            Console.WriteLine("Got uninitialized array");

            arr[0] = 0;
        }

        pool.Return(arr);
    });

    th.Start();

    th.Join();
}

Interestingly, a pool created using ArrayPool<T>.Create() works as expected.

So what does the "Shared" name mean in this context?

ArrayPool<T>.Shared is implemented by TlsOverPerCoreLockedStacksArrayPool, which maintains a pool per thread in order to avoid having to take a lock when renting/returning in the majority case. The comments explain the setup:

The implementation uses a tiered caching scheme, with a small per-thread cache for each array size, followed by a cache per array size shared by all threads, split into per-core stacks meant to be used by threads running on that core. Locks are used to protect each per-core stack, because a thread can migrate after checking its processor number, because multiple threads could interleave on the same core, and because a thread is allowed to check other core's buckets if its core's bucket is empty/full.

You can, as you discovered, create your own pool if that behavior is not desirable for your use case.

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

Non-actionable stale issue

Navigation Menu

Repository navigation

list of usage examples that illustrate current problems with ArrayPool and what the desired behavior should be #52098

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

list of usage examples that illustrate current problems with ArrayPool and what the desired behavior should be #52098

Description

Activity

jkotas commented on Apr 30, 2021

jkotas commented on Apr 30, 2021

jkotas commented on Apr 30, 2021

jkotas commented on Apr 30, 2021

Maoni0 commented on Apr 30, 2021

antonfirsov commented on May 3, 2021

davidfowl commented on May 3, 2021

Maoni0 commented on May 5, 2021

13 remaining items

antonfirsov commented on Jul 25, 2021

stephentoub commented on Jul 25, 2021

jkotas commented on Sep 24, 2021

Bio2hazard commented on Sep 25, 2021

tmds commented on Sep 27, 2021

dennis-yemelyanov commented on Feb 27, 2022

saucecontrol commented on Feb 27, 2022

dotnet-policy-service commented on Oct 19, 2026

dotnet-policy-service commented on Oct 19, 2026

jkotas commented on Oct 19, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions