Skip to content

HexEncodingProvider.TryEncode throws after partly writing the destination for inputs ≥ 1 GiB, and GetMaxEncodedLength returns a negative bound #150

Closed

Description

@matt-edmondson

What's wrong

Essentials.EncodingProviders.Hex/HexEncodingProvider.cs doubles lengths in three places:

  • L23: GetMaxEncodedLength(int sourceLength) => sourceLength * 2;
  • L33: if (destination.Length < data.Length * 2) return false;
  • L44: bytesWritten = data.Length * 2;

All three are unchecked int arithmetic, and the repo doesn't enable CheckForOverflowUnderflow. For any input of 1,073,741,824 bytes or more, data.Length * 2 wraps to a negative number. Two things follow.

The size guard never rejects. destination.Length < negative is false, so a destination that is too small is accepted. The loop at L38-42 writes into it until the span bounds check throws IndexOutOfRangeException. The caller gets an exception from a Try method instead of false, and the destination has already been partly overwritten. HexObfuscationProvider.TryObfuscate delegates here, so it behaves the same way.

The bound is negative. For example, GetMaxEncodedLength(int.MaxValue) returns -2.

  • IEncodingProvider.Encode(ReadOnlySpan<byte>) (Essentials/IEncodingProvider.cs ~L65) passes that value to ArrayPool<byte>.Shared.Rent, which throws an ArgumentOutOfRangeException about pool sizes rather than about the input.
  • Callers that size their own buffer from the bound get a negative size.

The other providers already handle this:

Suggested fix

Compute in long, as DeflateBound does:

  • GetMaxEncodedLength throws ArgumentOutOfRangeException when 2L * sourceLength > int.MaxValue.
  • TryEncode returns false when 2L * data.Length > destination.Length.

Acceptance criteria

  • GetMaxEncodedLength(int.MaxValue) throws ArgumentOutOfRangeException instead of returning a negative value.
  • TryEncode never throws for a destination that is too small, and leaves bytesWritten == 0. The guard can be tested without allocating 1 GiB, for example through GetMaxEncodedLength or an internal helper.
  • HexObfuscationProvider.GetMaxObfuscatedLength and TryObfuscate get the same behaviour through delegation.

Activity

  1. matt-edmondson commented on Oct 19, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions