Problem
ByteBoolEncodingDecoder (reader/decode/ByteBoolEncodingDecoder.java:20-33) repacks a byte-per-boolean input buffer into the bit-packed BoolArray layout, allocating and running a read-modify-write loop over every row:
long packedBytes = (n + 7) >>> 3;
MemorySegment packed = ctx.arena().allocate(packedBytes > 0 ? packedBytes : 1);
for (long i = 0; i < n; i++) {
if (bytes.get(ValueLayout.JAVA_BYTE, i) != 0) {
long byteIdx = i >>> 3;
byte cur = packed.get(ValueLayout.JAVA_BYTE, byteIdx);
packed.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) ((cur & 0xff) | (1 << (i & 7))));
}
}
return new MaterializedBoolArray(ctx.dtype(), n, packed);
The allocation is modest (n/8), but it is entirely avoidable, and the O(n) loop it exists to feed — a data-dependent branch plus a read-modify-write per row — is the more interesting cost. vortex.bytebool is the one boolean encoding where the source buffer is already directly indexable per row.
Fix
Add LazyByteBoolArray implements BoolArray — roughly ten lines, mirroring the other lazy carriers in reader.array:
public record LazyByteBoolArray(DType dtype, long length, MemorySegment bytes) implements BoolArray {
@Override
public boolean getBoolean(long i) {
Objects.checkIndex(i, length);
return bytes.get(ValueLayout.JAVA_BYTE, i) != 0;
}
}
and have the decoder return it, zero-copy over the mmapped buffer.
BoolArray.materialize(SegmentAllocator) already provides the bit-packing fallback for consumers that need a real LSB-first bitmap (reader/array/BoolArray.java:49-60) — it is the same loop this decoder currently runs eagerly, so nothing is lost, it just moves to the callers that actually want a bitmap.
Bounds note: the input buffer is untrusted and mmap-bounded, so it can be shorter than the claimed n. The current eager loop would fault on bytes.get(...) past the end; the lazy version needs the same treatment the sibling decoders apply — either a SegmentBroadcast.capacity check up front, or the broadcast-modulo convention the Materialized*Array family uses. Check what vortex.bool does and match it.
Context
Found in a sweep for remaining eager materializations after #329 / 7e0d6e7. This is one of the smaller items in that sweep — see the sibling issues for the vortex.runend string expansion, vortex.sequence, and the vortex.dict primitive path, which are the material wins.
Problem
ByteBoolEncodingDecoder(reader/decode/ByteBoolEncodingDecoder.java:20-33) repacks a byte-per-boolean input buffer into the bit-packedBoolArraylayout, allocating and running a read-modify-write loop over every row:The allocation is modest (
n/8), but it is entirely avoidable, and the O(n) loop it exists to feed — a data-dependent branch plus a read-modify-write per row — is the more interesting cost.vortex.byteboolis the one boolean encoding where the source buffer is already directly indexable per row.Fix
Add
LazyByteBoolArray implements BoolArray— roughly ten lines, mirroring the other lazy carriers inreader.array:and have the decoder return it, zero-copy over the mmapped buffer.
BoolArray.materialize(SegmentAllocator)already provides the bit-packing fallback for consumers that need a real LSB-first bitmap (reader/array/BoolArray.java:49-60) — it is the same loop this decoder currently runs eagerly, so nothing is lost, it just moves to the callers that actually want a bitmap.Bounds note: the input buffer is untrusted and mmap-bounded, so it can be shorter than the claimed
n. The current eager loop would fault onbytes.get(...)past the end; the lazy version needs the same treatment the sibling decoders apply — either aSegmentBroadcast.capacitycheck up front, or the broadcast-modulo convention theMaterialized*Arrayfamily uses. Check whatvortex.booldoes and match it.Context
Found in a sweep for remaining eager materializations after #329 / 7e0d6e7. This is one of the smaller items in that sweep — see the sibling issues for the
vortex.runendstring expansion,vortex.sequence, and thevortex.dictprimitive path, which are the material wins.