Skip to content

Reject unpaired surrogates in JsonWriter STRICT mode - #3121

Open
sushant-me wants to merge 1 commit into
google:mainfrom
sushant-me:fix/jsonwriter-unpaired-surrogates
Open

sushant-me wants to merge 1 commit into
google:mainfrom
sushant-me:fix/jsonwriter-unpaired-surrogates

Conversation

@sushant-me

Copy link
Copy Markdown

JsonReader rejects unpaired UTF-16 surrogate characters in strict mode (that was #3113 / #3116), but JsonWriter has no equivalent validation, so the two sides of the API disagree about what a strict JSON string is.

An unpaired surrogate in a Java String has no faithful JSON encoding: a JSON string is a sequence of Unicode characters (RFC 8259 section 7). Two consequences today, both silent:

  • JsonWriter with Strictness.STRICT writes the value anyway, producing a document that its own Javadoc promises conforms to RFC 8259 but which JsonReader in the same strictness mode then refuses to read back. Gson's own default output is not readable by Gson.
  • On the documented write-to-UTF-8 path the character becomes ?, so data is silently lost.

What this changes

JsonWriter.string(...) is the single sink for both names and values, so validating there covers value(String), name(String) and deferred names alike:

private void string(String value) throws IOException {
  validateString(value);
  ...

validateString mirrors JsonReader.validateString exactly, including the message, so a caller that catches one path understands the other.

Scope

The check is limited to Strictness.STRICT. LEGACY_STRICT and LENIENT are unchanged, so no existing caller sees different output. That mirrors how the read side was introduced, and is why this is not a behaviour change for anyone who did not already opt into strict mode.

Tests

Adds the writer-side coverage that never existed:

  • testStrictModeRejectsUnpairedSurrogates — lone high, lone low, embedded, and reversed pair.
  • testStrictModeRejectsUnpairedSurrogateInName — the name path (rejection surfaces when the entry is completed, because names are written lazily).
  • testStrictModeAllowsPairedSurrogates — a valid pair is written and then read back in strict mode, which is the round-trip property the bug broke.
  • testNonStrictModesStillAllowUnpairedSurrogates — pins the legacy behaviour so it cannot regress accidentally.

mvn -pl gson test passes: 4671 tests, 0 failures, 0 errors.

@google-cla

google-cla Bot commented Sep 15, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@sushant-me
sushant-me force-pushed the fix/jsonwriter-unpaired-surrogates branch from 4e519de to 77a9ba7 Compare September 15, 2026 04:45
@sushant-me

Copy link
Copy Markdown
Author

@googlebot I signed it!

JsonReader has rejected unpaired UTF-16 surrogates in STRICT mode since
PR google#3116, but JsonWriter still writes them. That produces a document which
this class's own documentation promises conforms to RFC 8259 but which no
conforming parser can read back, and which is silently replaced by '?' when
the document is encoded as UTF-8. Gson's own default output is therefore
unreadable by Gson in STRICT mode.

Validate the value on the write path the same way the read path does, so
the two sides of the API agree on what a strict JSON string is. The check is
limited to Strictness.STRICT so callers of the legacy permissive modes see
no behaviour change.

Adds writer-side tests mirroring the existing JsonReaderTest coverage; the
writer previously had none.
@sushant-me
sushant-me force-pushed the fix/jsonwriter-unpaired-surrogates branch from 77a9ba7 to 9706389 Compare September 15, 2026 08:59

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant