Repository navigation
Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque<usize> #15573
Description
Is your feature request related to a problem or challenge?
Both @rluvaton and I have seen
I saw this when running the following query
./datafusion-cli-filter-pushdown -c "SELECT \"WatchID\", \"ClientIP\", COUNT(*) AS c, SUM(\"IsRefresh\"), AVG(\"ResolutionWidth\") FROM hits WHERE \"SearchPhrase\" <> '' GROUP BY \"WatchID\", \"ClientIP\" ORDER BY c DESC LIMIT 10;"Using hits.parquet: https://datasets.clickhouse.com/hits_compatible/hits.parquet
I am not sure what @rluvaton was doing when he saw it
Describe the solution you'd like
Review and see if this is a performance bottleneck we can remove
Describe alternatives you've considered
No response
Additional context
No response
Activity
alamb commentedon Apr 3, 2025
BTW this code was added in
SortPreservingMergeExec#12302Rachelint commentedon Apr 4, 2025
take
alamb commentedon Apr 4, 2025
FYI @Rachelint I think @rluvaton has a proposal on a ticket as well
cloneonuninitiated_partitionsin SortPreservingMergeStream #15562