You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Reduce per-row sorting allocations in multi-entry map_sort normalization #5900
Normalizing multi-entry map<string,int> batches currently slices each row's keys and invokes Arrow sorting, allocating temporary indices and prefix tuples for every map. The singleton optimization in #5887 avoids this work for one-entry maps, but multi-entry maps still pay these costs.
Reuse an Arrow-compatible prefix-tuple scratch buffer across rows for multi-entry string-key/int-value maps, while preserving the singleton fast path, exact duplicate-key permutations, sliced offsets, null buffers, schema metadata, and unsupported-key errors. Also fill rebased offsets in bulk for all-empty visible batches.
Matched release microbenchmarks show about 3x faster normalization for 2–10-entry forward maps and 33–39% faster normalization for 2–50-entry maps in independent paired measurements. Validation covers normalization, hashing, combined execution, nulls, mixed cardinalities, Unicode keys, and fallback types.
Related to #5818 and the matched benchmark work in #5822.
Activity