Skip to content

JsonSerializationProvider.Deserialize(ReadOnlySpan<byte>) returns null for UTF-8 JSON that starts with a BOM (the YAML provider accepts it) #76

Closed

Description

@matt-edmondson

What's wrong

Essentials.SerializationProviders.Json/JsonSerializationProvider.cs:68 Deserialize<T>(ReadOnlySpan<byte> data) passes the bytes straight to JsonSerializer.Deserialize<T>(data, options). System.Text.Json's UTF-8 span overload does not skip a byte-order mark, so input starting with EF BB BF throws JsonException. The provider catches that and returns default.

A scratch app run against main (8546476) printed:

bytes = EF BB BF + {"A":1}
Json provider  Deserialize<Dictionary<string,int>>(bytes) -> null
Yaml provider  Deserialize<Dictionary<string,int>>(bytes) -> ok

Why it matters

File.ReadAllBytes on a file saved by Notepad, Visual Studio or PowerShell 5 (Out-File -Encoding utf8) produces exactly this input. The caller gets null, as if the data were empty or corrupt. Because failure is swallowed, nothing points at the BOM. The YAML provider handles the same bytes, so switching providers behind ISerializationProvider changes the result. The text-based paths (for example FileSystem/Temp persistence via ReadAllTextAsync) are unaffected because the reader strips the BOM.

Suggested fix

Strip a leading UTF-8 BOM before deserializing:

ReadOnlySpan<byte> bom = [0xEF, 0xBB, 0xBF];
if (data.StartsWith(bom)) data = data[bom.Length..];

Apply the same fix to any other byte-based deserialize path in the JSON provider (and check the NewtonsoftJson/Toml providers' byte overloads for parity).

Acceptance: a test that BOM-prefixed UTF-8 JSON deserializes to the same value as the unprefixed bytes, for the JSON provider and each other serialization provider's byte overload.

Activity

  1. matt-edmondson commented on Sep 30, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions