Skip to content

[DOCS] Add blog post: The Serialization Tax, and How Sedona Stopped Paying It - #3401

Merged
jiayuasu merged 3 commits into
apache:masterfrom
jiayuasu:blog/serialization-tax
Sep 25, 2026
Merged

jiayuasu merged 3 commits into
apache:masterfrom
jiayuasu:blog/serialization-tax

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

  • No. This is a documentation update; the PR title uses the [DOCS] prefix.

What changes were proposed in this PR?

Add the blog post “The Serialization Tax, and How Sedona Stopped Paying It,” with a cover illustration and a timing chart.

The post follows a transform-buffer-area query to explain how Sedona's Spark SQL expressions avoid serializing intermediate spatial results. It introduces SerdeAware, distinguishes geometry support in 1.4.0 from raster support in 1.4.1, and explains which expression and materialization boundaries interrupt direct object passing.

Measurements on Sedona 1.9.1 compare nested expressions with variants containing IF barriers, using 3.82 million Washington buildings, replicated county geometries, and generated rasters. The largest observed difference is 0.69 versus 1.24 seconds: the barrier query takes about 80% longer, equivalent to about 44% less time for the nested query. The post reports the control timings and treats the results as end-to-end query comparisons, since they do not isolate serialization cost.

How was this patch tested?

  • Checked the benchmark scripts and recorded results against the post and chart: equal aggregate outputs, retained barriers, timing calculations, data preparation, and non-null IDs.
  • Ran the existing SerdeAwareFunctionSpec: both tests passed using the existing compiled test classes. A separate diagnostic compiled the relevant current source and verified serializer counts for scalar results, geometry results, unsupported child expressions, and null handling.
  • Ran a small reproduction on Sedona 1.9.1 and Spark 3.5.4 to verify that the raster setup reads materialized cached inputs.
  • Documentation hooks and mkdocs build passed. The build emitted warnings for existing documentation outside this post; the rendered post and both figures were checked. The full performance benchmark was not rerun during this editorial review.

Did this PR include necessary documentation updates?

  • Yes. The change adds the post and its figures.

@jiayuasu
jiayuasu marked this pull request as ready for review September 25, 2026 07:27
@jiayuasu
jiayuasu merged commit aafe0d9 into apache:master Sep 25, 2026
11 checks passed
jiayuasu added a commit that referenced this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant