Skip to content

Add Stream wrappers for memory and text-based types - #126669

Merged
ViveliDuCh merged 23 commits into
dotnet:mainfrom
ViveliDuCh:stream-wrappers-api
Jun 24, 2026
Merged

Add Stream wrappers for memory and text-based types#126669
ViveliDuCh merged 23 commits into
dotnet:mainfrom
ViveliDuCh:stream-wrappers-api

Conversation

@ViveliDuCh

@ViveliDuCh ViveliDuCh commented Apr 8, 2026

Copy link
Copy Markdown
Member

Fixes #82801

Introduces standardized stream wrappers for text and memory types as public sealed classes (StringStream, ReadOnlyMemoryStream, WritableMemoryStream, and ReadOnlySequenceStream ) as approved in API review (video).

What's included

  • StringStream (System.IO, CoreLib): non-seekable, read-only stream that encodes string or ReadOnlyMemory<char> on-the-fly. Encoding is required (no default). Never emits BOMs.
  • ReadOnlyMemoryStream (System.IO, CoreLib): seekable, read-only stream over ReadOnlyMemory<byte>
  • WritableMemoryStream (System.IO, CoreLib): seekable, read-write stream over Memory<byte> with fixed capacity
  • ReadOnlySequenceStream (System.Buffers, System.Memory): seekable, read-only stream over ReadOnlySequence<byte>
  • Ref assembly updates for System.Runtime and System.Memory
  • Updated 3 internal call sites (ReadOnlyMemoryContent, XmlPreloadedResolver, JsonXmlDataContract) to use the new types
  • Conformance and behavioral tests for all 4 types

Implementation notes

  • All types are public sealed with direct constructors: the review moved away from factory methods on Stream to avoid derived-type IntelliSense noise
  • StringStream encodes directly into the caller's buffer in Read() rather than using an internal byte buffer
  • ReadOnlyMemoryStream and WritableMemoryStream derive from MemoryStream.
  • TryGetBuffer returns false and GetBuffer throws on the memory stream types (no publiclyVisible support for now)

Limitations

  • The internal shared-source ReadOnlyMemoryStream in Common/ cannot be fully replaced:
    • System.Memory.Data multi-targets netstandard2.0 where the new public type doesn't exist.
    • System.Net.Http (net11.0 only) can adopt it directly.

Follow-up work

  • IBufferWriter<byte>Stream wrapper (separate proposal per #100434)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces new standardized Stream wrapper types across CoreLib and System.Memory, aligning with the approved API shape from #82801 and updating a few internal call sites to use the new wrappers.

Changes:

  • Added new public sealed stream wrappers: StringStream, ReadOnlyMemoryStream, WritableMemoryStream (CoreLib) and ReadOnlySequenceStream (System.Memory).
  • Updated reference assemblies and project files to expose/compile the new APIs.
  • Added conformance + targeted behavioral tests, and updated select internal consumers to use StringStream.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Implements the new StringStream API (read-only, non-seekable, on-the-fly encoding).
src/libraries/System.Private.CoreLib/src/System/IO/ReadOnlyMemoryStream.cs Implements seekable read-only stream over ReadOnlyMemory<byte>.
src/libraries/System.Private.CoreLib/src/System/IO/WritableMemoryStream.cs Implements seekable fixed-capacity read/write stream over Memory<byte>.
src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems Wires new CoreLib stream wrapper source files into the build.
src/libraries/System.Private.CoreLib/src/Resources/Strings.resx Adds a new resource string entry (currently appears unused).
src/libraries/System.Runtime/ref/System.Runtime.cs Updates public API surface to include the new stream wrapper types (and a small ref signature normalization change).
src/libraries/System.Runtime/tests/System.IO.Tests/System.IO.Tests.csproj Includes the new test files in the System.IO test project.
src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamConformanceTests.cs Adds conformance coverage for StringStream (string + ReadOnlyMemory<char> overloads).
src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_String.cs Adds targeted StringStream(string, Encoding) behavioral tests.
src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_Memory.cs Adds targeted StringStream(ReadOnlyMemory<char>, Encoding) behavioral tests.
src/libraries/System.Runtime/tests/System.IO.Tests/ReadOnlyMemoryStream/ReadOnlyMemoryStreamConformanceTests.cs Adds conformance coverage for ReadOnlyMemoryStream.
src/libraries/System.Runtime/tests/System.IO.Tests/ReadOnlyMemoryStream/ReadOnlyMemoryStreamTests.cs Adds targeted ReadOnlyMemoryStream behavioral tests.
src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamConformanceTests.cs Adds conformance coverage for WritableMemoryStream (with some overridden tests).
src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamTests.cs Adds targeted WritableMemoryStream behavioral tests.
src/libraries/System.Memory/src/System/Buffers/ReadOnlySequenceStream.cs Implements ReadOnlySequenceStream over ReadOnlySequence<byte>.
src/libraries/System.Memory/src/System.Memory.csproj Includes the new ReadOnlySequenceStream source file in System.Memory.
src/libraries/System.Memory/src/Resources/Strings.resx Adds SR strings needed for stream-like exception messages in System.Memory.
src/libraries/System.Memory/ref/System.Memory.cs Adds ReadOnlySequenceStream to the System.Memory ref surface.
src/libraries/System.Memory/tests/System.Memory.Tests.csproj Adds tests for ReadOnlySequenceStream and references StreamConformanceTests.
src/libraries/System.Memory/tests/ReadOnlyBuffer/ReadOnlySequenceStreamTests.cs Adds targeted behavioral tests for multi-segment ReadOnlySequenceStream scenarios.
src/libraries/System.Memory/tests/ReadOnlyBuffer/ReadOnlySequenceStream.ConformanceTests.cs Adds conformance coverage for ReadOnlySequenceStream.
src/libraries/System.Private.Xml/src/System/Xml/Resolvers/XmlPreloadedResolver.cs Switches internal string-to-stream conversion to StringStream.
src/libraries/System.Private.DataContractSerialization/src/System/Runtime/Serialization/Json/JsonXmlDataContract.cs Switches internal string-to-stream conversion to StringStream.
Comments suppressed due to low confidence (1)

src/libraries/System.Private.DataContractSerialization/src/System/Runtime/Serialization/Json/JsonXmlDataContract.cs:38

  • Variable is still named 'memoryStream' but the implementation is now StringStream (not a MemoryStream). Rename the local to avoid implying the stream is seekable/in-memory-buffered (e.g., 'xmlContentStream').
            Stream memoryStream = new StringStream(xmlContent, Encoding.UTF8);
            object? xmlValue;
            XmlDictionaryReaderQuotas? quotas = ((JsonReaderDelegator)jsonReader).ReaderQuotas;
            if (quotas == null)
            {
                xmlValue = dataContractSerializer.ReadObject(memoryStream);

Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Runtime/ref/System.Runtime.cs
Comment thread src/libraries/System.Private.CoreLib/src/Resources/Strings.resx Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/ReadOnlyMemoryStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/ReadOnlyMemoryStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Memory/ref/System.Memory.cs
Copilot AI review requested due to automatic review settings April 9, 2026 08:01
@ViveliDuCh
ViveliDuCh force-pushed the stream-wrappers-api branch from 27761ff to b43698b Compare April 9, 2026 08:01
@ViveliDuCh
ViveliDuCh force-pushed the stream-wrappers-api branch from b43698b to 95a2989 Compare April 9, 2026 08:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 10 comments.

Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Runtime/tests/System.IO.Tests/System.IO.Tests.csproj Outdated
Comment thread src/libraries/System.Memory/src/System.Memory.csproj Outdated
Comment thread src/libraries/System.Memory/tests/System.Memory.Tests.csproj Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/ReadOnlyMemoryStream.cs Outdated
Copilot AI review requested due to automatic review settings April 13, 2026 19:23
@ViveliDuCh
ViveliDuCh force-pushed the stream-wrappers-api branch from 5e3f98c to fec7a59 Compare April 13, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

Comment thread src/libraries/System.Private.CoreLib/src/System/IO/ReadOnlyMemoryStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/StringStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/WritableMemoryStream.cs Outdated
Comment thread src/libraries/System.Private.CoreLib/src/System/IO/WritableMemoryStream.cs Outdated
Copilot AI review requested due to automatic review settings June 23, 2026 23:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 1 comment.

@SimonCropp

Copy link
Copy Markdown
Contributor

awesome. that .net preview will this be in?

SimonCropp added a commit to SimonCropp/Polyfill that referenced this pull request Jun 24, 2026
* Add Stream wrappers for memory and text-based types

Polyfills StringStream, ReadOnlyMemoryStream, WritableMemoryStream and
ReadOnlySequenceStream from dotnet/runtime#126669.

These types are approved for net11 but are not in the current net11 preview,
so they are gated on `#if FeatureMemory` and remain active on net11. When a
net11 SDK ships the real types, ApiBuilderTests.StreamWrapperBclDetectionTests
fails with the exact change to make in each file: flip the guard to
`#if FeatureMemory && !NET11_0_OR_GREATER` and add a TypeForwardedTo under
`#if NET11_0_OR_GREATER`. The same steps are documented atop each source file.

- ReadOnlyMemoryStream/WritableMemoryStream derive from MemoryStream and delegate
  to the MemoryStream(byte[],int,int,bool,bool) constructor, giving read-only /
  fixed-capacity semantics with GetBuffer throwing and TryGetBuffer returning
  false, without relying on MemoryStream internals unavailable on older TFMs.
- StringStream (non-seekable, never emits a BOM) and ReadOnlySequenceStream
  (seekable) derive from Stream.

Tests run on every framework where FeatureMemory is available; verified compiling
across all 22 TFMs and passing on net11.0 and net48.

* .
@jozkee

jozkee commented Jun 24, 2026

Copy link
Copy Markdown
Member

/backport to release/11.0-preview6

@github-actions

Copy link
Copy Markdown
Contributor

Started backporting to release/11.0-preview6 (link to workflow run)

@github-actions

Copy link
Copy Markdown
Contributor

@jozkee backporting to release/11.0-preview6 failed, the patch most likely resulted in conflicts. Please backport manually!

git am output
$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: First API proposal implementation
Using index info to reconstruct a base tree...
M	src/libraries/System.Memory/ref/System.Memory.cs
M	src/libraries/System.Memory/tests/System.Memory.Tests.csproj
M	src/libraries/System.Private.CoreLib/src/Resources/Strings.resx
M	src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems
M	src/libraries/System.Runtime/ref/System.Runtime.cs
Falling back to patching base and 3-way merge...
Auto-merging src/libraries/System.Memory/ref/System.Memory.cs
Auto-merging src/libraries/System.Memory/tests/System.Memory.Tests.csproj
Auto-merging src/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Auto-merging src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems
Auto-merging src/libraries/System.Runtime/ref/System.Runtime.cs
Applying: API replacements in production code
Applying: Implementation update based on latest API Review final consensus
Applying: Address PR feedback. MemoryStream base change. Spillover buffer for Convert edge case. Flush strategy update.
Applying: Address PR feedback
Applying: Address PR feedback: fix StringStream flush offset, WritableMemoryStream length/seek semantics, ReadOnlySequenceStream safety, and consolidate tests
Applying: Address PR feedback: remove redundant code, improve test coverage and structure
Using index info to reconstruct a base tree...
M	src/libraries/System.Memory/src/System/Buffers/ReadOnlySequenceStream.cs
M	src/libraries/System.Net.Http/src/System.Net.Http.csproj
M	src/libraries/System.Private.CoreLib/src/System/IO/WritableMemoryStream.cs
M	src/libraries/System.Private.DataContractSerialization/src/System/Runtime/Serialization/Json/JsonXmlDataContract.cs
M	src/libraries/System.Runtime/tests/System.IO.Tests/ReadOnlyMemoryStream/ReadOnlyMemoryStreamConformanceTests.cs
M	src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_Memory.cs
M	src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_String.cs
M	src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamConformanceTests.cs
M	src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamTests.cs
Falling back to patching base and 3-way merge...
Auto-merging src/libraries/System.Memory/src/System/Buffers/ReadOnlySequenceStream.cs
Auto-merging src/libraries/System.Net.Http/src/System.Net.Http.csproj
Auto-merging src/libraries/System.Private.CoreLib/src/System/IO/WritableMemoryStream.cs
Auto-merging src/libraries/System.Private.DataContractSerialization/src/System/Runtime/Serialization/Json/JsonXmlDataContract.cs
CONFLICT (content): Merge conflict in src/libraries/System.Private.DataContractSerialization/src/System/Runtime/Serialization/Json/JsonXmlDataContract.cs
Auto-merging src/libraries/System.Runtime/tests/System.IO.Tests/ReadOnlyMemoryStream/ReadOnlyMemoryStreamConformanceTests.cs
CONFLICT (content): Merge conflict in src/libraries/System.Runtime/tests/System.IO.Tests/ReadOnlyMemoryStream/ReadOnlyMemoryStreamConformanceTests.cs
Auto-merging src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_Memory.cs
CONFLICT (content): Merge conflict in src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_Memory.cs
Auto-merging src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_String.cs
CONFLICT (content): Merge conflict in src/libraries/System.Runtime/tests/System.IO.Tests/StringStream/StringStreamTests_String.cs
Auto-merging src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamConformanceTests.cs
CONFLICT (content): Merge conflict in src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamConformanceTests.cs
Auto-merging src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamTests.cs
CONFLICT (content): Merge conflict in src/libraries/System.Runtime/tests/System.IO.Tests/WritableMemoryStream/WritableMemoryStreamTests.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0007 Address PR feedback: remove redundant code, improve test coverage and structure
Error: The process '/usr/bin/git' failed with exit code 128

Link to workflow output

jozkee pushed a commit that referenced this pull request Jun 25, 2026
… types (#129811)

Backport of #126669 to
release/11.0-preview6. See that PR for full details.

Adds new public stream wrapper APIs (`StringStream`,
`ReadOnlyMemoryStream`, `WritableMemoryStream`,
`ReadOnlySequenceStream`).

Fixes #82801.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-preview7 milestone Jun 25, 2026
SimonCropp added a commit to SimonCropp/Polyfill that referenced this pull request Jun 27, 2026
…562) (#563)

* Align stream wrappers with the dotnet 11 implementation (follow-up to #562)

StringStream and ReadOnlySequenceStream diverged from dotnet/runtime#126669
in ways that hurt performance on the cases those types exist for. Bring them
in line and port the PR's tests.

StringStream
- Encode incrementally via a stateful Encoder (fast-path single-shot, spillover
  buffer for sub-scalar reads, final flush) instead of encoding the whole text
  into one buffer on first read. Peak memory is now bounded by the caller's read
  buffer rather than the full encoded length, matching the BCL.
- Span-based Encoder.Convert/Encoding.GetBytes on net core 2.1 / netstandard 2.1+,
  array-based equivalents on older TFMs.

ReadOnlySequenceStream
- Track a SequencePosition cursor alongside the absolute position so Read/ReadByte
  advance from the current segment instead of re-slicing from the start every call
  (was O(segments^2) for multi-segment sequences). Seek/Position reposition the
  cursor forward from the current spot, walking from the start only for back jumps.
- Override CopyTo/CopyToAsync to write segments directly to the destination.
  CopyTo is gated to net core 2.1 / netstandard 2.1+ (Stream.CopyTo(Stream, int)
  is not virtual on older TFMs); CopyToAsync applies everywhere.

ReadOnlyMemoryStream / WritableMemoryStream are unchanged: they are already at
parity for array-backed memory, and matching the BCL for native (non-array)
memory would require MemoryStream internals unavailable outside CoreLib.

Tests
- Port the explicit unit tests from the PR (StringStream encodings, surrogate
  pairs, chunk boundaries, GetMaxByteCount overflow guard, memory-slice/char-array
  ctors; WritableMemoryStream capacity/seek/overwrite cases) into the TUnit suite,
  plus ReadOnlySequenceStream CopyTo/CopyToAsync coverage.

Regenerated Split + assemblySize; public API surface unchanged. Verified across
all 22 TFMs (Consume) and passing on net10.0 and net48.

* Update Directory.Build.props
eiriktsarpalis pushed a commit that referenced this pull request Jul 15, 2026
Fixes #82801

Introduces standardized stream wrappers for text and memory types as
public sealed classes (`StringStream`, `ReadOnlyMemoryStream`,
`WritableMemoryStream`, and `ReadOnlySequenceStream` ) as approved in
[API
review](#82801 (comment))
([video](https://www.youtube.com/watch?v=T7xc7rARCEw&t=0h9m41s)).

### What's included

- `StringStream` (`System.IO`, CoreLib): non-seekable, read-only stream
that encodes `string` or `ReadOnlyMemory<char>` on-the-fly. Encoding is
required (no default). Never emits BOMs.
- `ReadOnlyMemoryStream` (`System.IO`, CoreLib): seekable, read-only
stream over `ReadOnlyMemory<byte>`
- `WritableMemoryStream` (`System.IO`, CoreLib): seekable, read-write
stream over `Memory<byte>` with fixed capacity
- `ReadOnlySequenceStream` (`System.Buffers`, System.Memory): seekable,
read-only stream over `ReadOnlySequence<byte>`
- Ref assembly updates for `System.Runtime` and `System.Memory`
- Updated 3 internal call sites (`ReadOnlyMemoryContent`,
`XmlPreloadedResolver`, `JsonXmlDataContract`) to use the new types
- Conformance and behavioral tests for all 4 types

### Implementation notes

- All types are `public sealed` with direct constructors: the review
moved away from factory methods on `Stream` to avoid derived-type
IntelliSense noise
- `StringStream` encodes directly into the caller's buffer in `Read()`
rather than using an internal byte buffer
- `ReadOnlyMemoryStream` and `WritableMemoryStream` derive from
`MemoryStream`.
- `TryGetBuffer` returns false and `GetBuffer` throws on the memory
stream types (no `publiclyVisible` support for now)

### Limitations

- The internal
[shared-source](https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/System/IO/ReadOnlyMemoryStream.cs)
`ReadOnlyMemoryStream` in `Common/` cannot be fully replaced:
- `System.Memory.Data` multi-targets `netstandard2.0` where the new
public type doesn't exist.
    - `System.Net.Http` (net11.0 only) can adopt it directly.


### Follow-up work

- `IBufferWriter<byte>` → `Stream` wrapper (separate proposal per
[#100434](#100434))

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[API Proposal]: Add Stream wrappers for memory and text-based types

8 participants