Is your feature request related to a problem or challenge?
TopK can be optimized by filtering on the max value before converting the arrays to row-format (which is slow).
Describe the solution you'd like
Create a filter based on arrow-rs kernel (lt / lt_eq) and filter the array based on this.
By making the early filter fast, we can avoid converting most of the values to RowFormat and speed up TopK.
Describe alternatives you've considered
No response
Additional context
No response
Is your feature request related to a problem or challenge?
TopK can be optimized by filtering on the max value before converting the arrays to row-format (which is slow).
Describe the solution you'd like
Create a filter based on arrow-rs kernel (
lt/lt_eq) and filter the array based on this.By making the early filter fast, we can avoid converting most of the values to RowFormat and speed up TopK.
Describe alternatives you've considered
No response
Additional context
No response