Repository navigation
[Variant] Benchmark and optimize BinaryView construction in VariantArrayBuilder #10621
Description
Is your feature request related to a problem or challenge?
VariantArrayBuilder::build converts accumulated buffers and offsets into BinaryViewArrays by replaying every offset through BinaryViewBuilder::try_append_view.
A TODO suggests constructing the views directly would be faster, but the existing benchmarks combine this work with JSON traversal and Variant encoding. There is no benchmark showing how much time this finalization step consumes.
Describe the solution you'd like
First add a targeted benchmark that isolates VariantArrayBuilder::build for large arrays, particularly many small Variant values where per-view overhead is likely visible.
Use that benchmark to establish a baseline. If the finalization cost is material, construct the views during append or build the final view buffer directly, and report the measured improvement.
Activity
cakeni commentedon Aug 11, 2026
I'd like to work on this. I'll first add a focused build benchmark, measure the baseline, and only optimize if the finalization cost is material.
cakeni commentedon Aug 11, 2026
take
sdf-jkl commentedon Aug 12, 2026
thanks @cakeni, I'll take a look at all of your PRs
perf(variant): build BinaryView arrays directly (#10640)
perf(variant): build BinaryView arrays directly (apache#10640)
perf(variant): build BinaryView arrays directly (apache#10640)
perf(variant): build BinaryView arrays directly (apache#10640)
perf(variant): build BinaryView arrays directly (apache#10640)
perf(variant): build BinaryView arrays directly (apache#10640)
perf(variant): build BinaryView arrays directly (apache#10640)