Skip to content

fix(c/driver/postgresql): encode Arrow JSON for JSONB COPY - #4505

Merged
lidavidm merged 1 commit into
apache:mainfrom
BOKJUNSOO:fix-postgresql-copy-arrow-json-to-jsonb
Jul 14, 2026
Merged

fix(c/driver/postgresql): encode Arrow JSON for JSONB COPY#4505
lidavidm merged 1 commit into
apache:mainfrom
BOKJUNSOO:fix-postgresql-copy-arrow-json-to-jsonb

Conversation

@BOKJUNSOO

Copy link
Copy Markdown
Contributor

Summary

PostgreSQL binary COPY represents a jsonb field as a version byte followed by
the JSON text. The PostgreSQL driver currently sends the physical string storage
of extension<arrow.json> unchanged, so PostgreSQL interprets the first JSON byte
as the version (for example, { becomes version 123) and rejects the row.

This adds a target-aware JSONB COPY field writer. When the Arrow source is
extension<arrow.json> and the existing PostgreSQL target column is jsonb, the
writer prefixes the JSON bytes with version 0x01 and includes that byte in the
field length. Other string inputs and PostgreSQL json targets continue to use
the existing writer.

The existing JSONB ingest regression test now verifies a successful round-trip,
including a SQL NULL value.

Testing

  • pre-commit run --files c/driver/postgresql/copy/writer.h c/driver/postgresql/postgresql_test.cc (all applicable C/C++ checks passed)
  • adbc-driver-postgresql-test (242 passed, 6 skipped)
  • adbc-driver-postgresql-copy-test (73 passed)
  • Loaded 100 extension<arrow.json> values from the reported sample Parquet file through the locally built driver into a PostgreSQL jsonb column without a compatibility shim (100 inserted and validated)

Depends on #4499 for resolving existing PostgreSQL target column types during
COPY writer selection.

Closes #3293

@BOKJUNSOO
BOKJUNSOO requested a review from lidavidm as a code owner July 12, 2026 16:17

@lidavidm lidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks pretty straightforward.

@lidavidm

Copy link
Copy Markdown
Member

CC @Mandukhai-Alimaa

@BOKJUNSOO
BOKJUNSOO force-pushed the fix-postgresql-copy-arrow-json-to-jsonb branch from 8f48924 to f6eda15 Compare July 14, 2026 04:33

@Mandukhai-Alimaa Mandukhai-Alimaa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@lidavidm
lidavidm merged commit a79b674 into apache:main Jul 14, 2026
81 of 87 checks passed
@lidavidm lidavidm added this to the ADBC Libraries 24 milestone Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Jsonb in postgres and extension<arrow.json> are not supported for writer

3 participants