Repository navigation
[EPIC] A collection of improvement for the performance for sort and compare and gc, etc #7802
Description
This epic is the collection for improving the performance for sort and compare. cc @Dandandan @alamb
Current sub-tasks:
- Optimize comparison kernels for inlined views #7621
- Clickbench microbenchmark spends significant time in memcmp for not_empty predicate #7766
- Change use of
inline_valueto inline it to a u128 #7743 - Perf: optimize sort string_view performance #7790
- Improve StringArray(Utf8) sort performance #7847
- Speedup fixed row encoding #7844
- Speedup sorting for inline views: 1.4x - 1.7x improvement #7856
- perf: speed up StringViewArray gc 1.4 ~5.x faster #7873
- Improve the performance for coalese with StringView
- Optimize partition_validity function used in sort kernels #7937
- Perf: improve sort via
partition_validityto use fast path for bit map scan (up to 30% faster) #7962 - Perf: optimize actual_buffer_size to use only data buffer capacity for coalesce #7967
Activity
alamb commentedon Jun 27, 2025
This is pretty amazing @zhuqi-lucas -- love it
zhuqi-lucas commentedon Jul 2, 2025
I will follow-up adding more subtasks:
Today plan is to experiment:
StringArray sort improvement
Dandandan commentedon Jul 2, 2025
Nice work @zhuqi-lucas
I plan to look at continuing #7844 somewhere soon-ish.
Might help a bit with the encoding speed of primitives.
zhuqi-lucas commentedon Jul 2, 2025
Interesting improvement @Dandandan !
zhuqi-lucas commentedon Jul 2, 2025
Added the two sub-tasks now, thanks!
Dandandan commentedon Jul 3, 2025
Another one #7856
zhuqi-lucas commentedon Jul 3, 2025
Thank you @Dandandan , added to sub-tasks, and reviewed the PR!
[-][EPIC] A collection of improvement for the performance for sort and compare[/-][+][EPIC] A collection of improvement for the performance for sort and compare and gc, etc[/+]zhuqi-lucas commentedon Jul 5, 2025
I also including GC improvement for this collection.
Try to add a sub task for gc:
#7873
zhuqi-lucas commentedon Jul 20, 2025
Continue adding more in progress tasks for this epic:
#7962
#7967
alamb commentedon Jul 21, 2025
Amazing -- thank you @zhuqi-lucas