Repository navigation
Use NullBuffer::union_many when appropriate #21880
Description
Is your feature request related to a problem or challenge?
Several DataFusion PRs (#21464, #21468, #21471, #21475, #21477, #21482, #21532) optimize NULL handling in scalar functions by replacing row-by-row null buffer construction with bulk NullBuffer::union. When 3+ null buffers need combining, they chain binary union calls, each allocating a new BooleanBuffer.
@mbutrovich added a new NullBuffer::union_many API in arrow 58.2.0 that avoids this allocation
Describe the solution you'd like
Use NullBuffer::union_many instead of NullBuffer::union when unioning together multiple buffers
Describe alternatives you've considered
No response
Additional context
No response
Activity
alamb commentedon Apr 27, 2026
Note we need arrow 58.2.0 to be released
I have a PR draft to do this, but I need to finalize the release first
mbutrovich commentedon Apr 27, 2026
Thanks for making an issue to revisit this, @alamb!
neilconway commentedon Apr 27, 2026
take
perf: Use `NullBuffer::union_many` (#22070)