[adamsitnik]

Recently System.IO.Tests have started failing with OOM on Unix-like OSes. Since the CI has not produced any dump file and according to my knowledge we don't offer any memory profiler for Linux, I've used VS Memory Profiler to profile System.IO.Tests on Windows.

The number of live objects goes back to previous value (or a very similar number) after each GC collection, so I don't think that we have introduced a memory leak to the product itself.

However, one of the tests, namely WriteChars_VeryLargeArray_DoesNotOverflow allocated 6.5 GB and it has already been causing OOMs in the past as it's marked to skip on Android:

runtime/src/libraries/System.IO/tests/BinaryWriter/BinaryWriter.EncodingTests.cs

Lines 191 to 192 in 57bfe47

	[SkipOnPlatform(TestPlatforms.Android, "OOM on Android could be uncatchable & kill the test runner")]
	public unsafe void WriteChars_VeryLargeArray_DoesNotOverflow()

I guess (I have no memory dump from the CI) that the problem is caused by WriteChars_VeryLargeArray_DoesNotOverflow which allocates 6.5GB and causes other tests which run in parallel to fail with OOM.

My proposal:

• allocate less memory to validate Int32Overflow, just slightly more than int.MaxValue
• since we don't validate the buffers content (just the stream position) use the same memory for input and output
• don't run other tests in parallel
• throw SkipTestException on OOM so we can differentiate successful and skipped test

This is nicely reducing the amount of memory used by the System.IO.Tests and hopefully is still testing what author wanted to test (cc @GrabYourPitchforks)

fixes #65791

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Recently System.IO.Tests have started failing with OOM on Unix-like OSes. Since the CI has not produced any dump file and according to my knowledge we don't offer any memory profiler for Linux, I've used VS Memory Profiler to profile System.IO.Tests on Windows.

The number of live objects goes back to previous value (or a very similar number) after each GC collection, so I don't think that we have introduced a memory leak to the product itself.

However, one of the tests, namely WriteChars_VeryLargeArray_DoesNotOverflow allocated 6.5 GB and it has already been causing OOMs in the past as it's marked to skip on Android:

runtime/src/libraries/System.IO/tests/BinaryWriter/BinaryWriter.EncodingTests.cs

Lines 191 to 192 in 57bfe47

	[SkipOnPlatform(TestPlatforms.Android, "OOM on Android could be uncatchable & kill the test runner")]
	public unsafe void WriteChars_VeryLargeArray_DoesNotOverflow()

I guess (I have no memory dump from the CI) that the problem is caused by WriteChars_VeryLargeArray_DoesNotOverflow which allocates 6.5GB and causes other tests which run in parallel to fail with OOM.

My proposal:

• allocate less memory to validate Int32Overflow, just slightly more than int.MaxValue
• since we don't validate the buffers content, only stream position, use the same memory for input and output
• don't run other tests in parallel
• throw SkipTestException on OOM so we can differentiate successful and skipped test

This is nicely reducing the amount of memory used by the System.IO.Tests and hopefully is still testing what author wanted to test (cc @GrabYourPitchforks)

fixes #65791

Author:	adamsitnik
Assignees:	-
Labels:	area-System.IO
Milestone:	7.0.0

[adamsitnik]

/azp run runtime-libraries-coreclr outerloop-linux

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[adamsitnik]

I've created new issues for the CI failures: #66395 #66396

[danmoseley]

Thanks for taking care of this @adamsitnik