Skip to content

Use Stream.CopyTo{Async} in NonCryptographicHashAlgorithm.Append{Async} - #103669

Merged
stephentoub merged 1 commit into
dotnet:mainfrom
stephentoub:hashcopyto
Jun 19, 2024
Merged

stephentoub merged 1 commit into
dotnet:mainfrom
stephentoub:hashcopyto

Conversation

@stephentoub

@stephentoub stephentoub commented Jun 18, 2024 •

Copy link
Copy Markdown
Member

Rather than using a manual loop to copy data from a stream to a buffer, use the Stream.CopyTo{Async} method. Downside is there's an extra 32 byte allocation for the temporary destination passthrough Stream object. Upside is it allows the source Stream's to optimize via its CopyTo{Async} override, e.g. MemoryStream.CopyTo just makes a single Write call with its buffer. The base Stream.CopyTo also uses a larger rented buffer size than these APIs were using.

Related to #103539

Before:

Method Kind Mean Allocated
Hash MemoryStream 79.76 us 568 B
Hash FileStream 837.08 us 568 B

After:

Method Kind Mean Allocated
Hash MemoryStream 59.76 us 600 B
Hash FileStream 131.51 us 600 B
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.IO.Hashing;
using System.Security.Cryptography;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[MemoryDiagnoser]
[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
public class Tests
{
    private Stream _stream;
    private byte[] _bytes = RandomNumberGenerator.GetBytes(1024 * 1024);

    [Params("MemoryStream", "FileStream")]
    public string Kind { get; set; }

    [GlobalSetup]
    public void Setup()
    {
        switch (Kind)
        {
            case "MemoryStream":
                _stream = new MemoryStream(_bytes);
                break;

            case "FileStream":
                string path = Path.GetRandomFileName();
                File.WriteAllBytes(path, _bytes);
                _stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 0, FileOptions.DeleteOnClose);
                break;
        }
    }

    [GlobalCleanup]
    public void Cleanup() => _stream.Dispose();

    [Benchmark]
    public ulong Hash()
    {
        _stream.Position = 0;
        var hash = new XxHash3();
        hash.Append(_stream);
        return hash.GetCurrentHashAsUInt64();
    }
}

@stephentoub
stephentoub merged commit f8e575c into dotnet:main Jun 19, 2024
@stephentoub
stephentoub deleted the hashcopyto branch June 19, 2024 23:46
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-hashing, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

@github-actions github-actions Bot locked and limited conversation to collaborators Jul 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants